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Overview 


Quality of online learning is a priority for the Kingdom of 
Saudi Arabia. As the COVID-19 pandemic and public 
health concerns continue to require emergency remote 
instruction and learning online across the globe, 
educators and students continue to persevere through 
the new normal acquiring new skills to teaching and 
learning online and accessing technological and support 
resources to assist them. To ensure that high quality 
online learning is being offered in the future, the 
Kingdom of Saudi Arabia commissioned phase two of a 
study to understand the state of online learning and to 
gauge the satisfaction of stakeholder groups: students, 


parents, teachers, and administrators. 


The intention is to benchmark satisfaction over time while also 
determining challenges and opportunities for improvement while 
marking areas of excellence that could be highlighted against an 
evaluation framework for quality online learning. The findings and 
recommendations of this report can provide strategic guidance to 
help decision making to enhance programming within the Kingdom 
of Saudi Arabia to improve the overall student learning experience. 
MM alicuiw oy-Tan Cele] t-laNvar-]co(-tol opm nat-M[arellUii(olaKolm-lar-] NM nikon tal=) 
Madrasati learning management system (LMS) tool that supports 
successful digital learning in remote and online contexts. 
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Executive Summary 


The Kingdom of Saudi Arabia swiftly moved its K-12 education programs and 
services online in response to the COVID-19 pandemic. To ensure quality online 
learning was offered, the Kingdom commissioned a developmental study to 
understand the state of online K-12 education during the fall 2020 semester 
and begin visioning the post-COVID environment. The intention was to 
determine opportunities for improvement while identifying areas of excellence 
that could be highlighted against an evaluation framework for quality K-12 
online learning. As an outcome of the initial study, key recommendations were 
made. As a follow-up, this developmental study was conducted to analyze the 
progress made on the recommended initiatives and measures the satisfaction 
of key stakeholders, challenges, as well as the opportunities for future 
investment against the dimensions of that National eLearning Center’s (NELC) 
comprehensive online learning evaluation framework. 


The Phase | study chiefly affirmed that the pandemic required rapid acceleration in 
the development and delivery of fully online courses for a high volume of students. 
These requirements were piled atop the needs already articulated in the nation’s 
ambitious Vision 2030 agenda for education. The Saudi Ministry of Education quickly 
extended its research focus to all areas of the K-12 education sector in an attempt to 
drive deeper into how to improve outcomes around key areas of online learning 
quality associated with Vision 2030. Based on Phase | findings and recommendations, 
more than 70 new initiatives were launched and implemented by the Ministry. 


A total of 387,957 members of the Saudi Arabia education community (students, 
parents, teachers, and administrators) participated in this follow-up study. Data was 
collected through qualitative and quantitative surveys. Updated themes and trends 
emerged from this study and provide a clear picture of the strong progress made. 
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Key Themes and Areas of Notable Progress 


Teaching and Learning 


Design, navigation, and organization of the classes 
Content and materials in the classes 

Assessments, feedback, and grading of student learning 
Accessibility 

Online teaching 

Online learning 

Collaborations and interactions among student peers 
Student engagement and learning 


Communication 
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Communication and interactions between students and 
teachers 

School communication with students 

School communication with parents 

Teacher communication with parents 


Technology and Human Infrastructure 
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Educational technologies used in classes 
Technology support and resources 
Learning support and resources 

Training and orientation to online learning 
Teaching support 


Leadership 


Policy 
Operations 


High Level of Satisfaction and Progress Indicated 

Overwhelmingly students, parents, teachers, and principals report satisfaction with the online 
learning in response to COVID-19 global pandemic resulting in emergency remote instruction and 
learning at a distance. Notably, from Phase | to Phase Il, all four stakeholder groups participating 
in the study demonstrated a substantial increase in overall satisfaction, with each group reporting 
satisfaction rates above 80%. In particular, students’ satisfaction nearly doubled during Phase Il, 
from 43% in Phase | to more than 85%. 
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The majority of students, parents, teachers, and principals reported satisfaction with their 
school’s online education when asked about their overall satisfaction. Parents were also satisfied 
with learning support and resources, as well as communication from teachers and schools. 
Students and parents also felt strongly about students' active involvement in classes and the 
amount of work they completed. They reported strong agreement in statements regarding the 
quality of the student experience online and the learning opportunities presented to them by their 
teachers. In fact, across every dimension of the NELC Online Learning Evaluation Framework, 
stakeholder responses (where applicable) indicated satisfaction above 50%, and frequently far 
above. In this way, the entire survey indicates broad support and satisfaction for the efforts within 
the Kingdom in responding to COVID-19, as well as support for expansion of online learning 
beyond the pandemic. Further, internal benchmarking across demographic groups within the 
data reveal internal consistency of support and satisfaction with current online learning efforts. 


K-12 Phase | vs Phase II Overall Satisfaction 


88.6% 90.7% 


84.6% 86.0% 
62.0% 65.5% 
| l - 5 
Teachers Administrators Students Parents 
m@ Phase II 88.6% 90.7% 84.6% 86.0% 
m@ Phase | 62.0% 65.5% 43.0% 41.0% 
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Acknowledgement of Online Learning Efficiency 


Additionally, all stakeholder groups believed that classes were run more efficiently by the teachers 
after moving to an online learning environment. Stakeholders felt that their online and onsite 
experiences are comparable and recommend continuing online learning. Respectively, majority of 
students report that online classes have comparable outcomes to onsite classes, learning that 
takes place online is comparable to learning that takes place in the onsite classes, they learn as 
much in my online classes as they would have if the class had been offered face-to-face, and that 
they are able to successfully complete all of their work after moving online. Parents do report that 
online classes have comparable outcomes to onsite classes and that learning that takes place 
online is comparable to learning that takes place in the onsite classes. 
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Improved Digital Literacy Skills 


Among the most promising areas of growth and satisfaction regarding online learning from an 
administrator perspective during the COVID-19 pandemic concerned the remote shift’s impact on 
digital literacy (use of digital tools, ability to work in digital environments). An impressive 93.9% of 
Administrators identified improvement in students’ skills and 95.7% of administrators saw 
improvement in teachers’ skills. These indicators are strong and suggestive of the opportunity of 
digital learning to continue informing and playing a role in the holistic education landscape of 
Saudi Arabia in the post-pandemic world. 


Online Learning as a Strategic Choice for the Future 

The results from evaluation and continuous improvement survey indicators suggest there is a 
body of reflective practitioners overseeing learning inclusive of digital modalities. These educators 
collected data and reported satisfaction with student performance in the online medium. 
Educators were generally not opposed to the expansion of online learning and perhaps most 
important, the majority of students and parents would recommend that their school strategically 
expand online learning in the future, and if given the opportunity, students report that they look 
forward to learning online again. Over 80% of administrators and teachers indicated that their 
strong agreement that if given the opportunity, they would look forward to offering fully-online 
instruction at their schools again. 


Madrasati as a Leader of Internally-Developed Platforms 

Given that the most of K-12 education in the Kingdom went online using the Madrasati Learning 
Management System (LMS) Platform during the fall 2020 semester, an additional study 
component sought to measure the satisfaction of LMS use during Phase | & Il. Findings from the 
study report high degrees of satisfaction across stakeholder groups. 


Historically, education ministries 
and governments across the 
international community tended to 
formally adopt LMS less frequently 
at the national levels, often as a 
measure of cost savings and 
practical implementation. While 
decisions to adopt LMS in K-12 have 
most often been made at a local 
level rather than at a state or 
national level, the pandemic did 
spur some nations to establish LMS 
or contract with commercial 
vendors to expand access toonline | 
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learning. Furthermore, many nations sprang into action to support continuity in learning for 
students by establishing content and resource material repositories through a distribution model. 


The establishment of the Madrasati LMS stands out in the domain of global education, both in 
terms of its robust capabilities for delivering online learning, as well as its proliferation across the 
entirety of the K-12 education spectrum in KSA. In the analysis that follows, the features of 
Madrasati lead the way for internally developed LMS platforms among other nations, and 
Madrasati compares favorably with the leading commercial LMS providers. This analysis explored 
the e-learning solutions developed as a response to COVID-19 in approximately 174 countries. 
After review of the international landscape, Madrasati is relatively peerless related to other 
nations’ efforts to implement a countrywide LMS for e-learning in the K-12 education sector. 


Challenges and Opportunities for Investment 

Although there were high levels of satisfaction for all components of online learning in survey 
instrumentation, the lowest area for students was technology support and resources. Teachers 
and parents, although above average levels of agreement, also report slightly lower levels for 
technical support. Principals, too, report lower levels of agreement with statements regarding 
availability of technical staff to support the learning environment and the technology needs of 
teachers. While principals felt that there are adequate policies and procedures to support online 
learning, developing new policies and providing resources to ensure technology access, including 
laptops and Internet access, as well as appropriate support are critical to future strategic planning 
to ensure quality. Teachers report that a top area investment should be made in teacher 
professional development training and teacher incentives. 


Recommendations 
To ensure quality in online education, the findings and expert review of this body of data informa 
series of propositions to guide decision-makers and suggest resources for future strategic 
planning and continuous improvement efforts. Among these findings are: 
e Ensure students have access to a laptop, Internet, and a supportive study environment. 
e Provide strategies for skill building for students and parents in time management, 
organization, and technology work productivity applications through an online learning 
readiness intervention. 
e Implement professional development for online teaching with teacher incentives to 
improve use of time and engagement of students. 
e Update policy, regulations, and standards to support high quality online learning. 


Recommendations for future developments for the Madrasati platform and online learning in KSA 
include mobile device support, nationwide broadband infrastructure and device support 
expansion, adaptive learning and analytics support, and expanded teaching training and support. 
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This study was coordinated by The National eLearning Center (NELC) of the Kingdom of Saudi Arabia, in 
partnership with the Online Learning Consortium (OLC). NELC was established as an independent entity 
by the Council of Ministers of Saudi Arabia, and aims to enhance trust in eLearning programs, lead 
innovation in learning and digital transformation, and to enable the integration among educational 
institutions and labor market needs in the Kingdom of Saudi Arabia. The OLG, established in 1999 by the 
Sloan Foundation, is a collaborative community of higher education leaders and innovators, dedicated to 
advancing quality digital teaching and learning experiences designed to reach and engage the modern 
learner - anyone, anytime, anyplace. 


/n partnership with the Online Learning Consortium (OLC), a global leader with a vast network and 
expertise, this Phase |! Developmental Study was launched in late 2020 and early 2021. OLC is the lead of 
this project with assistance of several key partners from around the world. The study’s international 
advisory board included the UNESCO Institute for Information Technologies in Education, the World Bank, 
the International Council for Open and Distance Education (ICDE), the International Society for 
Technology in Education (ISTE), the Association for Learning Technology (ALT), European Distance and 
eLearning Network (EDEN), and the DETA Research Center. Partners informed the study from launch to 
conclusion advising on the study design and recommendations. 


Reviewing Organization: International Council for Open and Distance 


OER Learning (ICDE) Reviewer: Ebba Ossiannilsson 
ADVOCACY 


COMMITTEE 


Reviewing Organization: International Society for Technology in Education 
Reviewer: Brandon Olszewski 


Reviewing Organization: UNESCO Institute for Information Technologies in 
Education, Reviewer: Natalia Amelina 
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Introduction 


In the midst of a global pandemic that is slowly moving to the post-inoculation phase, 
students and educators are actively participating in remote emergency instruction and 
learning while slowly gaining the appropriate knowledge, skills, and resources to be 
successful in this new normal. 


The commitment to learning during a global pandemic has taken an emotional and financial toll on 
students, parents, teachers, and administrators. While all stakeholder groups are working to improve 
learning experiences and to ensure strategic planning in allocating resources and services to move 
towards high quality online learning, the Kingdom of Saudi Arabia has commissioned this Phase II 
Developmental Study to understand the state of online learning during COVID-19 to gauge stakeholder 
satisfaction across the areas of the quality framework, including leadership, design and planning, online 
teaching and learning, technology, training and support for students and teachers, and continuous 
improvement. 


In initial efforts, the Kingdom of Saudi 
Arabia commissioned a study to 
understand the state of online learning 
pre-, peri- and post-COVID. The 
intention was to determine 
opportunities for improvement while 
identifying areas of excellence that 
could be highlighted against an 
evaluation framework for quality 
online learning. Using this information, 
all K-12 education programs within the 
Kingdom of Saudi Arabia would be 
positioned to make improvements to the overall student learning experience. With the COVID-19 
pandemic disrupting education across the globe in early 2020, many educators found themselves rapidly 
transitioning to remote learning without the appropriate knowledge, skills or resources. 


The current study highlights current practices, informs strategic decision making, guides allocation of 
resources to overcome barriers to quality, and identifies investments to enhance the quality of the online 
learning experience for students. Using this information, schools within the Kingdom of Saudi Arabia will 
be positioned to make improvements driven by the data, research, and findings of this study to improve 
students’ learning experiences, increase instructional effectiveness of teachers, and ensure efficient and 
effective operations and support services for schools. Investments will be strategic and data driven to 
overcome barriers to online learning excellence. 
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The report has two primary components and 
includes several appendices. The first section 
introduces Phase II Developmental Study 
research framework concerning online 
learning satisfaction and benchmarking, as 
well as its findings and recommendations for 
future online teaching and learning in the 
Kingdom of Saudi Arabia. The second section 
details the context for and nationwide launch 
of the Madrasati LMS across K-12 schools, as 
well as benchmarking across the 
international community and 
recommendations for future LMS 


387,957 


innovations. 
Total surveys accessed 
Total voices influencing Within each section, contexts, frameworks, 
EEA of K-12 education and findings are detailed, as well as 


recommendations based on the findings 
herein. Collectively, these research entities 
connected with a total of 387,957 members of 
the Saudi Arabia education community”. 


With regards to this report’s structure, each 
component is similar. This introduction 
provides context for the study. An executive 
summary provides an overview of the areas of 
study. The context and framework section for 
each situates this research into earlier 
findings from Phase | into COVID-19 
educational response in the Kingdom of Saudi Arabia. This also details the organizing principles of quality 
online education to which schools and institutions are presently aligning and measuring their work. 


The methodology and findings section includes a summary of the analysis of each aspect of the surveys 
administered in the study. The recommendations section provides an interpretation of the findings and 
propositions. Finally, the next steps closing discusses a summary of overall findings and the future of 
education holistically and as a result of this incredibly rich compilation of research. 


It is of chief strategic advantage that research and analysis into the satisfaction and comparative 
outcomes of online learning generally occurred at the same time as the Kingdom of Saudi Arabia’s (KSA) 
Madrasati learning management system (LMS) international benchmarking study. As an LMS is a primary 
organizing tool and driver of the online learning experience, the features and status of nationally 
established and supported e-learning technology platforms for K-12 education offer incredible insight into 
the efficacy of digital learning. 


* 191,308 reviewed and provided feedback in some form in support of the Online Learning Satisfaction and Benchmarking Study commissioned by NELC and the OLC and 196,649 participated in 
the LMS Study sponsored by the Digital Transformation Unit of the Saudi Arabia Ministry of Education. 
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As ministries of education and governing bodies across the world have expanded their e-learning 
resources and supports for teachers and students as a result of the COVID-19 pandemic, so has the need to 
understand the effectiveness and scale of the efforts nations have made. KSA’s Madrasati platform, which 
has been internally developed within the country and its education ministry, has been launched and 
scaled in ways that differentiate it from other internally developed and commercially developed LMS 
platforms. Thus, discussion of the Madrasati platform seeks to benchmark KSA’s progress in launching the 
Madrasati, as well as a comparison to responses across the globe. 


It should be noted that the KSA’s response to 
the COVID-19 pandemic, particularly in the 
realm of virtual learning toward e-learning has 
not emerged wholly sui generis. Efforts to 
support high quality education and e-learning, 


have been a strategic emphasis for the GCF W abeKommmerolaalpaliansvemnem-) allan) elinlelers 
Kingdom of Saudi Arabia for some time. As the peu 
OECD (2020) noted in its report Education in cross-sectoral reform agenda In Its 
Saudi Arabia, KSA has “committed to an VATS (0) 040110) goals to promote alelaakelal 


ambitious cross-sectoral reform agenda” in its 
Vision 2030 goals to promote human capital, 


capital, with education leading the 
with education leading the way toward amore | \\E\V/\ We) ger Maale) a= prosperous 


prosperous economy (p. 3). Thus, even prior to 
the ongoing global COVID-19 pandemic and 
disruption caused by the rapid transition to 
online and distance learning, Saudi Arabia was 
innovating with its emphasis on online 
learning. As an example, KSA has launched 
iEN, a broadcast network that includes more 
than 23 channels that provide ubiquitous 
educational content and programming for students across the nation. Over the past year, many of these 
lessons have been hosted on YouTube for asynchronous viewing and access. The Ministry of Education has 
also begun to digitize curriculum across the country, enabling teachers and students to access that 
content via computing devices and smartphones. Broadly, students, teachers, supervisors, school 
libraries, and learning resource centers have all benefited from these efforts. 


economy 
-(p. 3). OECD (2020), Education in Saudi Arabia 


These efforts to expand e-learning have continued during the pandemic, particularly in relation to the 
LMS. For the beginning of the 2020 school year, KSA launched the Madrasati (“My School” in Arabic) 
learning management system to better facilitate virtual learning. In its first semester of launch, the use 
data from the Ministry of Education shows more than 6 million students and more than half a million 
educators using the platform to provide e-learning amid the pandemic (Saudi Ministry of Education, 
2020). The platform is an internally developed LMS and bears much of the same functionality and 
performance that most education authorities find in outside vendors and commercial LMS providers. 
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Analysis of the Madrasati platform within the context of international LMS adoption during COVID-19 
seeks to understand the following outcomes to better inform educators, administrators, and ministry 
leadership in further development of quality e-learning for K-12 education in KSA: 


e Increase understanding of the changes taking place in K-12 education post-pandemic across 
the globe; 


e Provide evidence of the progression of institutional and instructional practices in response to 
COVID-19 with strategic forethought to provide access, support, and instruction to students 
remotely; 


e Create a comparative analysis of the national platform in the Kingdom of Saudi Arabia as 
compared to international platforms; and 


e Report on differences in technological infrastructure functionality, use of technological features, 
and the influence of technology infrastructure on administration, and teaching and learning 
practices. 
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Context and Framework 


Purpose and Background 


This report represents Phase II of an ongoing strategic partnership and research commitment to study, 
understand, and continuously improve the state of online learning in the Kingdom of Saudi Arabia during 
and after COVID-19, as well as a benchmarking analysis on the Madrasati LMS implemented nationwide in 
K-12 schools for the Fall 2020 Semester. The Phase | report published in October 2020, The State of Online 
Learning in the Kingdom of Saudi Arabia: A COVID-19 Impact Study for K-12, shared a data-driven analysis 
of stakeholder perceptions across the Kingdom of Saudi Arabia’s K-12 education sectors in order to 
identify areas of excellence and determine opportunities for improvement. 


Findings from the first study indicated the pandemic required rapid acceleration in the development and 
delivery of fully online courses across for a high volume of students. In addition to existing needs related 
to the nation’s ambitious Vision 2030 agenda for education, findings also further illuminated critical needs 
to support success. The Ministry quickly extended its research focus on all areas of the K-12 education 
sector to hone in on key discoveries from Phase | and push deeper analysis into how to improve outcomes 
around key areas of online learning quality associated with Vision 2030. 


The Phase | Development Study areas of interaction and improvement helped refine the NELC Online 
Learning Evaluation Framework, which organizes analysis of the elements of quality online learning. This 
framework has further been used to group and organize the research design, instrumentation, and 
subsequent findings of both the COVID-19 Phase | and Phase II Development Study. Quantitative and 
qualitative survey measurements were taken and findings were analyzed thematically with respect to the 
evaluation framework. The dimensions of online education quality when grouped accordingly helps 
measure performance tied to recognized online education best practices. The framework is further 
defined, applied, and aligned in quality standards measuring multiple applications of online learning for 
institutional, programmatic, and school-level self-assessment and continuous improvement strategic 
planning through the leadership of the National eLearning Center (NELC) in Saudi Arabia. 
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NELC’s Online Learning Evaluation Framework 


Leadership 


Governance, Strategies, Policies, Processes, Resource 
Allocations, Periodic Review & Updating 


Curriculum Design & 
Planning 


Instructional Design Methods & UDL, Alignment with 
Standards, Course Syllabi, Course Materials & Content 


Online Teaching & 
Learning 


Communication, Engagement, Expectation Setting, 
Outcomes, Course Interaction, Feedback, Resources 


Assessment Strategies, Assessment Processes, 
Assessment Methodology, Assessment Types 


Technology 


Operability, Centralized Online Education Infrastructure 
(SSO, LMS, etc.), Modality, Security, ITSM Compliance, 
Internet Access, Reliability, Coverage 


Student Support 


Student Orientation & Support, Equity, Accessibility, 
Compliance Standards 


) 
) 
) 
fae ) 
) 
) 
) 


Training Support 


Technical Assistance, Professional Development, 
Orientation, Mentoring 


Evaluation & Continuous y 
Improvement 


Student Satisfaction, Teacher Satisfaction, Staff 
Satisfaction, Evaluation of Course Outcomes and 
Program Quality 


Based on the findings and recommendations from Phase | of the study, more than 70 new initiatives have 


been launched and implemented by the Ministry of Education. See Appendix A for a full listing of these 
initiatives and their correspondence to the framework elements above, as well as their implementation 


status at the time of this publication. 
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K-12 Initiatives Achievement 


K-12 Study - Phase | 
Level of Achievement for Initiatives 


Total of Initiatives: 71 


In Progress (20) 
28% 


Done (51) 
72% 


= Done (51) In Progress (20) 


Number of Initiatives According to Dimensions 


Online 


: Curriculum Learning Student Orientation Continuous 
LecHersnip Design and Aesessmett PechnGlogy Support | and Training Development Ue 
Teaching 
byl 
20 
lal 7 6 {Al 


Total tf 6 aly 6 17 


) Research Context 


Leadership Readiness 
The first key dimension of the initial study centered Di Taatclakielamelats 
on leadership readiness for digital learning with 

particular emphasis on pre-planning and response to 


the COVID-19 global pandemic. Overall findings 
indicated strong agreement from administrators, staff, and teachers that leadership subdimensions of 


governance, strategy, and policy are in place to support quality online learning throughout the Kingdom. 
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These findings were further supported by the OECD report How the COVID-19 Pandemic is Changing 
Education: A Perspective from Saudi Arabia, in their finding that schools were well-positioned to manage 
the pandemic, though internet access was a concern. With respect to processes, resource allocation, and 
innovation, there was also broad agreement from these stakeholders, yet an opportunity to more 
equitably allocate resources, pursue process improvement, and open innovation opportunities. Such 
questions were mined explicitly across K-12 stakeholder groups in this research to surface progress. 


Curriculum Design and Planning 

The second dimension of this research concerned 
curriculum design and planning. This area mined into Dimension Two 
all key aspects of digital learning design and pedagogy 
which precedes any aspect of instructional delivery. 
Specifically, instructional design methods such as universal design for learning (UDL), use of explicit 
learning objectives, alignment, course materials and content, and innovation were reviewed. A high 
degree of readiness regarding curriculum design and planning was observed in overall survey results, 
potentially due to existing efforts to increase online education in K-12 environments as well as the 
curriculum responsibilities of the Ministry. Study results did suggest that despite this readiness prior to 
the pandemic, the shift to emergency remote teaching revealed pedagogical considerations across this 
dimension that were not evident, or perhaps not necessary for curriculum prior to COVID. Survey 
recommendations included the opportunity to more explicitly support student readiness and continuous 
improvement given a broader scale of digital learning use and involvement for schools, teacher, and 
students alike. While much progress has been made, these factors remain items keyed in on by 
stakeholders as continuing opportunities within the Phase II] Development Study’s recommendations. 


Online Teaching and Learning 

The Online Teaching and Learning dimension of Phase | 
research measured seven subdimensions including 
Communication, Engagement, Expectation Setting, 
Outcomes, Course Interaction, Feedback, and 
Innovation. Readiness in online teaching and learning, like previous dimensions, reflected preparedness 
pre-COVID, with both increases in some areas post-COVID that indicated prompt attention to needs and 
new needs illuminated by the pandemic. Of particular note regarding a need for increased readiness 
moving forward in this dimension were course interaction and feedback. Findings related to online 
teaching and learning indicated that there were slight increases in the communication, engagement, and 
course interaction subdimensions, and slight decreases in outcomes and feedback post-COVID. These 
elements are critical to online teaching and learning, and these slight increases suggest that stakeholders 
may have made changes to support students online following the pandemic. Recommendations included 
a number of items that support continuous improvement in online teaching and learning, and provided a 
solid foundation for increasing effectiveness and capacity headed into this Phase Il research. Data-driven 
decision making was among the chief factors cited as a recent emphasis and success amidst the global 
pandemic. Continued investment in resources that amplify continuous improvement cycle processes for 
schools and institutions and the professional development that arms teachers with deepened skill sets 
and fluency with digital pedagogy are of paramount importance. 
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Assessment 

The assessment dimension measured four subdimensions 5 : 
including assessment strategies, assessment processes, DyTaatsakirelammelele 
assessment methodology, and innovation. Findings related 
to readiness for assessment in a fully-online environment 
indicated overall agreement, though at a lower rate and with more neutral responses than other aspects 
of the initial research. Survey and interview results showed that there was agreement with regard to the 
communication, understanding, and implementation of online assessment. However, teachers were not 
prepared to complete course assessments in this environment, and results further suggested that in order 
to ensure readiness for effective assessment practices online, clear expectations and agreement should be 
established regarding strategies, processes, and methodology to adequately measure learning. Phase Il 
research explicitly tackled unanswered questions associated with initial assessment impressions 
including teacher and administrator concerns about academic integrity in online assessments, teacher 
advocacy for online assessment procedures, and progress towards online exams that would cement 
greater confidence and alignment in assuring and measuring students’ digital learning outcomes. 


Technology 

Core to successful digital learning is planning for and the 
readiness of a strong technology backbone. This Di reatclaksilela Five 
dimension measured centralized online education 
infrastructure, operability, modality, security, information 
technology service management (ITSM) compliance, 
internet access, reliability, coverage, and innovation. Prior to COVID-19, there were clear needs related to 
technology that were planned for in long-range online education plans. Post-COVID, demands on 
technology increased, and survey and interview results showed that although the areas represented in the 
technology dimension were not fully prepared for the shift, improvements were and continue to be 
planned and implemented. Of particular note in this sub dimension, increasing the capability of 
centralized infrastructure to manage high-volume use, ensuring security related to student data and 
privacy, and making improvements to internet access, reliability, and coverage, including both network 
and device challenges, were all needed to ensure readiness in upcoming terms. Consistent through Phase 
Il analysis, the areas of internet access, reliability, coverage, and operability remain challenges and 
understandably so as plans and efforts to scale up are midstream. These remain vital as without an 
accomplished technology disposition across all subdimensions, students may lack the resources needed 
for successful remote learning. 


Student Support 

With perhaps equal importance to what is actually taught 
online (digital pedagogy) are considerations of and 
provision of online student services. These reinforce, 
maximize, and facilitate quality online and digital learning 
and stem the digital divide from students working and learning in a remote fashion. Considerations 
include student orientation and support, equity, accessibility, compliance, social emotional learning, and 
innovation. In Phase | of research, overall findings indicated a high level of awareness among 
administrators, staff, and teachers regarding holistic student support needs. Both quantitative and 
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qualitative data findings showed that while there were clear needs for improvement in all subdimensions, 
robust on-site services were previously implemented and there are innovative ideas and processes in 
development to extend these support services to students in a digital environment. 


~~ In the further development and implementation 
of appropriate and effective student support, the 


most evident gaps were between administrators, 
staff, and teachers and parents/students. 
Additionally, there was a need for increased 
assessment of needs and communication to 
parents and students regarding the availability of 
support. Some schools had already engaged in 
addressing these needs, and the 
recommendations for this have carried forward 
to presently recommended solutions for further 
development and improvement. 


Training and Support 

The Training and Support dimension concerns areas 
including technical assistance, professional development, 
orientation (Student & Parent), mentoring, and innovation. 
Training and support readiness appears to have been 
commensurate with the pace of online learning development and delivery in a pre-COVID environment. 
Post-COVID, sufficient resources exist for prior needs and actions have been taken to build upon training 
and support initiatives continued or launched during COVID. Overall findings indicated that school 
administrators, staff and teachers were satisfied with the relative activities and opportunities provided. 
School administrators, staff and teachers agree or strongly agree, suggesting continued focus and 
attention in this area are on track. 


Dimension Seven 


Evaluation and Continuous Improvement 
Finally, the Evaluation and Continuous Improvement ie - : 
dimension delves into the evaluation of course outcomes Dimension Eight 
and program quality, student satisfaction, teacher 
satisfaction, staff satisfaction, and innovation. 
Respondents indicated that improvements to readiness are needed regarding evaluation and continuous 
improvement. Areas requiring the most attention following Phase | concerned the evaluation of course 
outcomes and program quality, including the development and implementation of clear processes and 
metrics, and process implementation of student satisfaction measures. Both teacher and staff satisfaction 
increased following COVID-19, though recommendations were provided to further improve the ability to 
collect and constructively use feedback. Survey questions focused on evaluation and continuous 
improvement were included for administrators/staff, teachers, parents and students. Qualitative data 
from interviews was also obtained from administrators/staff and teachers. These efforts have been 
extended through Phase Il, leading to specific and updated findings and recommendations. 


CONSORTIUM, 
) Scope and Methodology 


Phase II Developmental Study 

Against the backdrop of the Phase | study and findings and with respect to the NELC Online Learning 
Evaluation Framework for measuring quality online and digital learning adopted by the Ministry of 
Education in the Kingdom of Saudi Arabia, this Phase II report and study addresses stakeholder 
satisfaction across multiple pinpointed areas of the quality framework, assessing updated status and 
continued needs and focus regarding leadership, design and planning, online teaching and learning, 
technology, training and support for students and teachers, and school and institutional continuous 
improvement. 


Drawing on the eight dimensions and subdimensions of 
the NELC Evaluation Framework and previous data, four 


new descriptive surveys were developed... 


The recommendations resulting from Phase | survey analyses provided information to support capacity 
and quality in online education throughout the country to enhance the potential for Vision 2030 
excellence in a diversified and knowledge-based economy. Drawing on the eight dimensions and 
subdimensions of the NELC Evaluation Framework and previous data, four new descriptive surveys were 
developed for K-12 administrators, teachers, parents, and students by DETA and OLC in November and 
December 2020. 


Survey instrumentation was developed based on a scan of other instrumentation and tools, including 
national and international products, literature, and recent research relevant to the dimensions and 
subdimensions. Survey items were developed to measure the participants’ attitudes, opinions, and beliefs 
about online learning at their institutions, to assess the performance of the institutions and different 
functions of the institutions to support online learning and quality, and to identify areas of challenge and 
opportunities for improvement in future academic terms. All survey items are considered to have both 
construct and content validity. 


The survey was delivered through NELC Survey Tool and was designed as a single instrument with 
branching logic to ease data collection and dataset management. Items were individually coded by 
stakeholder group and dimension for analysis and storage. Survey distribution was requested through the 
Ministry of Education in January and February 2021, who provided the survey link to other administrators, 
staff, teachers and students. Survey analysis commenced in February and March 2021 and included 
descriptive statistical analysis completed in SPSS, disaggregated by stakeholder group 
(administrators/staff, teachers, and students) as well as by dimension. 
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Learning Management System (LMS) 

For the LMS benchmarking section, data compiled and analyzed through this project are sourced from 
publicly available datasets, secondary analysis of publicly available reports and analyses, and academic 
and popular press research on comparative and international education and educational technology 
implementation, as well as educational technology market reporting. Data reporting also comes from 
press releases, white papers, and social media posts from commercial vendors and official government 
publications. In total, this analysis explored the e-learning solutions developed as a response to COVID-19 
in approximately 174 countries. 


It is worth acknowledging at the outset that much of the reporting in this analysis is based on available 
insights primarily from wealthier or more developed nations, where the technological and governing or 
educational infrastructure was able to provide or procure e-learning solutions that included LMSs as a 
component of their transition to remote teaching and learning. In many countries, the extent to which e- 
learning solutions were made available to students is incomplete. Where developed national response 
plans have been made available, many countries’ early responses to the pandemic indicated that 
searching and planning for national e-learning platforms was underway but were not yet operational. 


This analysis also includes information on strategic guidance for LMS implementation and usage, as well 
as promotional materials, from a number of LMS providers and vendors. All information from these 
analyses is used for benchmarking and comparison purposes and should not be intended as an 
endorsement for any particular learning management system, nor its parent company. 
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Participant Profile 
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As part of survey development, several dimensions for each stakeholder group were identified as valuable 
for gathering more in-depth and nuanced information, and survey questions were created for open-ended 
response. Qualitative interview participants included 19 administrators, 17 teachers, and 13 parents. NELC 
staff translated open-ended responses for OLC staff. This qualitative feedback was coded using case 
classifications to identify participant characteristics and gather baseline data. In-text coding was 
completed by dimension and sub dimension to provide detailed information about participants’ attitudes, 
opinions, beliefs, and experiences with interview questions. 


Quantitative and qualitative results 
were analyzed to develop 
recommendations aligned to the 
NELC Online Learning Evaluation 
Framework and organized by 
dimension and sub dimension, and 
to identify target audience(s), 
readiness, needs, initiatives, goals, 
actions, critical success factors, and Me Rd ‘&, 
key performance indicators. As is the Wh S 
hope of any research effort, strong 4 
synergy between feedback and data 
collected on the qualitative and ——— — ————™ | 
quantitative analysis was detected = ™ 
making this alignment fairly direct a ~ E.. 
and actionable. 
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fi This K-12 Phase Il Developmental Study analyzed responses from four different stakeholder 


groups (administrators, teachers, parents, and students) in response to surveys specific to 
each respondent. 


Key themes from their perspectives are reported below and aligned to the areas of the NELC Online 
Learning Evaluation Framework. Preceding the disaggregated analysis, however, must be the chief and 


principal finding that satisfaction with online learning in the context of the global pandemic was 
extremely high across all stakeholder groups. 


Strongly Neither agree nor 
= Agree (= Peres disagree 
[] Disagree [) Strongly Disagree 


Administrators 


49.86 40.86 5.64%! 2.26 


Teachers 


45.34 43.3% 7.24% | 2.6% | 
1.51% 


Parents 


43.7% 40.23 8.03% | 4.1% 
3.95% 


Students 


54.64 29.99 7.36%! 3.79 
4.22% 


The majority of students, parents, teachers, and 
principals reported satisfaction with their school’s 
online education when asked about their overall 
satisfaction. When asked about their general 
satisfaction, students and parents were primarily 
satisfied with their learning experience (83.9% parents 
and 84.6% students). The majority of students report 
that the were satisfied or very satisfied when reporting 
on their perceptions of educational technologies used in 
classes (87.2%), technology support and resources, 
learning support and resources, training and orientation 
to online learning, content and materials in the classes, 
assessments, feedback, and grading, online teaching, 
online learning, communication and interactions 
between students and teachers, collaborations and 
interactions among student peers, school 
communication with students. 


Parents were also satisfied with learning support and 
resources, as well as communication from teachers and 
schools, though satisfaction with teacher 
communication with parents was lower than other 
areas. Further, parents were satisfied with the learning 
experience their children received, including 
communication interactions between students and 
teachers, assessments, feedback, and grading. Parents 
were also satisfied with the design, navigation, and 
organization of the classes, as well as with the content 
and materials in the classes. 


Both students and parents were satisfied with the 
accessibility of the courses. While a very small 
percentage of accommodations were reported as 
needed, survey results from principals and teachers 
show that it may be an area for future investment. 


When prompted with a series of statements to gauge 
their level of agreement about their experiences with 
online learning, the majority of students reported 
satisfaction in teaching and learning online. These 
findings were particularly strong with respect to the 
technologies available to them, provided support for 


using technology and learning, instructional effectiveness and quality of their online classes, and learning 
effectiveness in online classes. Survey items related to instructional effectiveness in quality pedagogy and 
online instruction included active learning, assessment and feedback, clarity of instruction and learning, 
and interaction with content and course materials. 

Student Satisfaction 


81.8 


68.7 


Government Private International Schools Abroad 


Parent Satisfaction 


85.1 


Government Private International 
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Benchmarked across school types, students largely reported satisfaction with online learning. Notably 
within the student groups, both Governmental (84.5%) and Private school (81.8%) school students 
reported satisfaction above 80%, while International (65.4%) and abroad (68.7%) school students 
reported lower satisfaction. Nevertheless, these results demonstrate a strong majority of students across 
the groups reported satisfaction with online learning. Parent benchmarking across school types yielded 
similar results to the student satisfaction rates. Parents of Governmental school (85.0%) and Private 
school (85.1%) were nearly proportionately identical, while Private school parent satisfaction (79.5%) was 
just slightly lower. 


Overall, !am satisfied with my 


school’s online education. 


Related to benchmarking within stakeholder groups, survey respondents showed very similar satisfaction 
rates regardless of school types. Teacher respondents all showed very high satisfaction rates across 
school types, with Governmental (88.1%), International (91.7%) and Private (89.7%) school teachers all 
reporting near or above 90% satisfaction with online learning. Of the five Royal Commission for Jubail and 
Yanbu teachers surveyed, 100% reported satisfaction with online learning. 


Internal benchmarking across administrator respondents’ school groups also showed extremely high 
rates of satisfaction with online learning. At Governmental schools, satisfaction was 91.5%, while 
International school administrators reported 91.0% satisfaction. Private school administrators reported 
89.6% satisfaction, rounding out the three largest stakeholder groups. Of the four abroad school 
administrators, 100% of them reported satisfaction with online learning. 


Across school types for all four stakeholder groups, International schools were the only category to show 
much disparity between roles, as they showed the largest variation between school employees and 
students and/or their parents. In this way, teacher and administrators reported satisfaction above 90% on 
both occasions, while student satisfaction was 65.4% and parent satisfaction was 79.5%. Across other 
school types, the satisfaction data was relatively similar. 


Students and parents also felt strongly about students’ active involvement in classes and the amount of 


work they completed. They reported strong agreement in statements regarding the quality of the student 
experience online and the learning opportunities presented to them by their teachers. 
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Dyantlatsiiela My classes were run more efficiently by my teachers after moving to an 
online learning environment during COVID-19. 


Summary by 
Stakeholder 


85.13%, Students 


Parents 


| was able to successfully complete all of the work of my classes after 
Agree and moving online. 


Strongly Agree 87.26%) Students 


Responses 
Parents 


| felt | learned as much in my online classes as | would have if my class 
had been offered face-to-face. 


66.82% Students 
Parents 


Leadership 


Dimension The school's response to the pandemic supports quality online 
Summary by education and academic continuity. 


Stakeholder 


Agree and 
Sine) NVa-vageemem | would recommend that my school strategically expand online 
learning opportunities in the future. 


WK =}-(e a=) 65 


Responses 


Teachers 


As governance, strategies, policies, processes, and resource allocations are essential for creating 
conditions that frame a school or institution’s online experience, the leadership dimension’s results were 
clearly derivative of online satisfaction at-large. Administrators and teachers were similar in their 
agreement that moving to an online learning environment during COVID-19 improved the institutional 
culture around digital learning (93.9% agreement among administrators; 93.1% agreement among 
teachers). Administrators overwhelmingly found that schools provided policies and procedures to ensure 
quality online education (89.1%) and that teachers were more efficient operationally after moving to an 
online learning environment during COVID-19 (93.4%). Further, more than principals surveyed 
overwhelmingly reported satisfaction with the operations taking place at their school (93.8%). 
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Among the most promising areas of growth and satisfaction regarding online learning from administrator 
perspective concerned the COVID-19 pandemic and the remote shift’s impact on digital literacy (use of 
digital tools, ability to work in digital environments) with over 93.9% of administrators identifying 
improvement in students’ skills and 95.7% of administrators seeing improvement in teachers’ skills. 


Broadly, teacher perspectives 
on leadership were 
overwhelmingly positive. 


From the results of the survey, mmmMAlem ey-]ae(-lan](omce) ceccle MUR nom ULY=m- male) 


more than 91% of teachers : c 
sie isin an nat=inareye Moya (=r-\o(-1e-101] oMm [amr-(eiem lm 0) co)V(-Le li Ke) 


response to the pandemic all of us our capabilities in planning, and 
supported quality online 


education, and more than 90% mmc] JIUM4Kcm Kom eN=1qeke) gal =m- 11 tale meuale | (cratexoce 
The distribution of roles was really 


reported their school’s 
response supported academic 


continuity. Further, morethan [MiMJag) eXe)a-] alam kelomm weMe)V(-1 cece) ta(-B-] 1 als 
94% of teachers were satisfied b [ d | ° d 

with the communication from fo) oj ¢- Tol] oh ge) ems In order to 
Wis Leena oe succeed and achieve our goals. 

majority of teachers reported . 

that school leadership — Abullah Alha rbi, School Leader 
provides policies and 

procedures to ensure quality 
online education (87.4%) 


While student data did not specifically focus on school leadership, students did report strong support for 
their school strategically expanding online learning in the future (84%). Implicitly, combined with the 
overall satisfaction reporting, this finding speaks to the impact of leadership across students’ 
experiences. 


From parents’ perspectives, the survey data indicated that leadership supported quality learning 
experiences and academic continuity during the pandemic. To that end, the vast majority of parents 
reported that school leadership supported quality online education and academic continuity (84.1%). 
Furthermore, parents reported strong satisfaction with the communication they received from their 
child’s school (87.6%). These measures indicated that leadership did create conditions for quality 
learning and continuity throughout the pandemic, and that they communicated effectively with parents 
through a rapidly changing context. 


Curriculum Design & Planning 
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Related to curriculum design and planning, survey data indicates that administrators believed the 
transition to online learning creates quality experiences for students. Further, results also indicate that 
teachers had robust technology and media development support from staff in designing their instruction 
and classes. From teachers’ perspective, results related to curriculum design and planning were largely 
similar, with a large majority of teachers finding satisfaction with both the design of the curriculum and 


the support staff availability for the development of instruction. The table below displays results of this 
dimension for both groups. 


D)Taatsyarsiceyal Design of the curriculum, instruction, and learning supports quality 
Summary by online education. 
Stakeholder | 

Agree and 


Strongly Agree 
Responses 


Administrators 
Teachers 
Sufficient support staff are available to help the teacher in the 


designing of their instruction and class based on quality, including 
technology and media development. 


| Administrators 
Teachers 


Further related to curriculum and design, a significant finding from the data showed that more than 94% 
of teachers reported providing students with rich, current, and interactive content and media (94.1%), and 
nearly nine-in-ten teachers reported satisfaction with the quality of the content and materials for their 
classes (89.1%). For administrators, satisfaction with the quality of content and materials was similarly 
high (87.2%). 


Student data revealed strong satisfaction from students related to the curriculum design and planning. 
Across the survey items, students reported agreement or strong agreement that assignments were a fair 
gauge of what was learned, that teacher expectations were communicated clearly, that learning 
experiences featured rich content and media, that teachers used technology effectively, and that the 
students themselves were actively involved in their classes. 


From parents’ perspectives, the survey data strongly indicated that the curriculum design and planning 
led to high quality online learning experiences for their students. As displayed in the table below, more 
than three-quarters of parents reported satisfaction across each survey item related to curriculum design 
and planning. Results largely tracked with student results across this dimension, and indicate that the 
transition to online learning during the pandemic led to quality learning experiences. 


Assignments are a fair gauge of what is learned. 
836%) Students 
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Dimension Parents 


Su mmary by Teacher expectations are clearly communicated and resources are 
Sh) <alalelle(-jamm provided to complete required activities. | 
84.3% | Students 


Parents 


Agree and _ — 
Strongly Agree The class is rich, current, and interactive in content/media. 


Responses 77.1% | Students 
Parents 
Teachers use technologies effectively in my classes. 

84.21% | Students 


Parents 


| | am actively involved in my classes. | 
87:56%) Students 


Parents 


Online Teaching and Learning 


In addition to curriculum design and planning, the instructional delivery of online teaching and learning is 
an essential component of NELC’s Online Learning Evaluation Framework. At times online providers 
experience an implementation gap between what the instructional design intended and what is delivered 
to students, thus revealing opportunities for continuous improvement. This Phase II survey sought to 
more deeply interrogate the extent to which implementation gaps may have existed, and moreover how 
the implementation of online teaching led to increased success across areas including classroom/teacher 
communication, learner engagement, teacher expectation setting, course outcomes, learner interaction 
and feedback, and instructional innovation. 
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Dimension i ; . : 
Teachers were more efficient operationally after moving to an online 
Summary by learning environment during COVID-19. 


Stakeholder Administrators 94.5% 


Agree and Teachers 


Strongly Agree 
Responses Curriculum, class, and instruction are designed and updated based on 
data, evidence, and standards. 


Administrators 76.6% 


Teachers 


Administrators reported being very satisfied with online teaching and learning amidst the rapid ramp up 
to extensive digital learning during the COVID-19 pandemic. Students and parents were most satisfied 
with the student experience in the classes. Students and parents affirmed this, reporting a strong sense 
that students are actively involved, frequently interact, and complete their work. Teachers, 74.5% of 
whom agreed that they were chiefly involved in the design of curriculum, class, and instruction also 
agreed and reported that students are overall engaged in their online classes and interact frequently with 
teachers and other students. 


Teacher perspectives are particularly germane to this category of online learning quality given the brunt 
of day-to-day responsibilities that fall on their position in the area of instructional delivery. From the data, 
a total of 92.3% of teachers reported effectively using technology in their classrooms, while 95.2% of 
teachers said they communicated frequently and in a timely manner with students. Further, 82.3% 
reported that active learning was used in their classrooms as a tool to maximize student engagement. 


C C Online learning made me more creative and 
capable. The first thing | had to do was to 
understand the student’s strongest points and 
weakest points. | studied the learner's needs in 
order to best support them. 


--Noha Alrefaiy, Teacher 9 9 
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Teachers also overwhelmingly felt that students were engaged in their classrooms and interacted 
frequently with them and other students (89.8%). This result may connect to responses showing that 
teachers believed they managed students’ expectations with clear communication about the details of 
the class through an online class site and related helpful course materials (95.9%). Thus, related to 
managing expectations, teachers’ responses aligned with parents and students. Results indicated that 
teachers were managing students' expectations, clearly communicating the details of the class through an 
online class site, providing helpful materials, including rich and current content and media, 
communicating frequently and timely with students, and providing frequent feedback through 
assessments to improve student learning. 


aM at=m oy-larelslanlcomel(e Mm alelm-ixe) ome MmOlamua(-Mae)alie- lava 
WM koto) @-LeNV/-Jale-y:x-Me)mdal-malaat-w-]alem nore). 
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learning. Students took the exams and attended 
the classes as if nothing changed. 
--Misahall Al-Otaibi, Teacher 


Dimension The teacher sends or posts messages frequently and responds in a 
Summary by timely fashion. 


Students 80.06% 
Stakeholder 
85.01% | 


Agree and 
Strongly Agree The learner is connected in classes and interacts frequently with other 
students. 


Responses 


“A Students 90.99% 
CG (rer 88.43% 
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In addition to the findings above, survey data revealed that parents expressed strong agreement 
regarding their children’s learning and interaction within their classes. They reported that their child is 
connected in their classes, interacts frequently with other children, and is actively involved in their 
classes. They also reported the teacher’s expectations were clearly communicated to their child on the 


class site, resources were provided to help them complete the activities as required, teachers used 
technologies effectively, and the teacher sent messages frequently and responds to my child in a timely 


fashion. 


The Ministry of Education succeeded with 
online learning, and it succeeded on 
monitoring, managing, and supporting online 
learning. This led to positively impacting 
learning process for students. In this way, | 
believe online learning was great. 

— Waleed Al Shuayb, Parent 


Overall, the majority of students were satisfied with online teaching and online learning. From this 
analysis, few implementation gaps between the design and the delivery were found through the data. 
From the study, four high-level findings emerged, related specifically to online teaching and learning. 


Engagement on Track 

Students reported that they are engaged in their courses. They 
are actively involved in their classes. The assessments are active 
and provide them feedback on their learning. They report that 
they complete assessments frequently and receive feedback so 
that they can improve their learning as well as feel that their 
assignments are a fair gauge of what they have learned. 


Clear Expectations Communicated 

Students reported that they understand what is expected of 
them and have the resources they need to succeed. The 
teacher’s expectations were clearly communicated to them on 
the class site, and resources were provided to help them 
complete the activities as required. 
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Student and Teachers Socially Involved 
Students were socially involved with their teachers and their 
peers. Students reported that the teachers posted messages 
frequently and responded in a timely fashion. They reported 
social presence as well as feeling connected in their classes and 
interacting frequently with other students. 


Course Materials Calibrated 

Students reported that the content and course materials were 
effective. They agreed that they had rich, current, and 
interactive content and media in their classes. 


The alignment of perspectives across this category is salient to note. Students reported satisfaction with 
their schools’ and teachers’ abilities to ensure learning and instructional effectiveness through content, 
assessments, and interaction. Moreover, students were satisfied with content and materials in the classes, 
assessments, feedback, and grading, communication and interactions between students and teachers, 
collaborations and interactions among student peers, and school communication with students. Wholly, 
this suggests students and teachers were putting in great effort to learn and teach online, and all 
stakeholders were satisfied with the progress in the last year. 


Assessment 


Related to assessment, satisfaction data from all 
stakeholder groups was notably high. Principals and 
staff found strong satisfaction with the assessment 
of student learning (81.6%), while teacher 
satisfaction with assessment was even higher 
(86.3%). From students’ perspective, assessments, 
feedback and grading were highly satisfactory 
(88.0%), while parents’ sense of satisfaction was also 
markedly strong (86.3%). Across stakeholder groups, 
assessment was strong. One prominent finding from 
the study, related to the ways in which teachers 
provided assessments and communicated with 
students to improve learning. Of the more than 
34,160 teacher respondents, more than 95% 
reported providing low-stakes assessments and 
feedback to students in order to improve their learning (95.1%). Further, a similar percentage of teachers 
reported frequent and timely communication with students (95.2%), as managing students’ expectations 
and clearly communicating the details of the class to students (95.8%). What these results suggest is that 
teachers were incredibly dedicated to serving students and providing quality online learning experiences. 
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Dylaatalaksicela | provide students low-stakes assessments and feedback so that they 
' can improve their learning. 
iti 
95.1% 


| communicate frequently and timely with my students. 

95.2% 
| manage students’ expectations and clearly communicate the details 
of the class through an online class site and other helpful class 


materials. 
95.8% 


From students’ and parents’ perspectives, assessments were a similarly satisfactory component of the 
online learning experience, related to teachers’ and administrators’ perspectives. As shown in the table 
below, both students and parents reported high levels of satisfaction with both the frequency and 
feedback for learning. Similarly, both groups reported high levels of satisfaction with the assessments, 


feedback, and grading overall. 


Dimension Assessments are completed frequently and receive feedback to 
Summary by improve learning. 


Stakeholder Students 82.1% 
Parents 302% | 


Agree and 


Strongly Agree 
Responses 


=p Students 88.0% 
( (CG ) =} re nts 86.3% 


Satisfaction with assessments, feedback, and grading. 
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Technology 


Students and parents were generally satisfied and in agreement with the school’s ability to provide 
educational technology, technology support and resources, and learning support and resources, including 
training and orientation to online learning. 


Student Digital Devices 


@ Mobile Phones 
= Computers 


B Tablets 


It is important that students not only have technology resources needed for online learning, but that they 
have support for using technology and academic support online. Students reported that there were 
educational technologies available to help them learn online, that teachers used the technologies 
effectively, and that they were provided information on technology and learning support resources. 
Teachers were in agreement that they are using technology effectively as well. More specifically, students 
reported having access to appropriate support personnel to address student technical and class-related 
questions, problem reporting, and complaints. They were provided orientations to their online classes, 
the class design, and the technologies used in the class. The large majority of students felt that they had 
technical and academic support when they needed it. Regarding the technology, students and parents 
were Satisfied with their students’ learning management system and iEN TV experiences. 


Parents did report future recommendations for investments in broadband internet for their students 
(67.7%), as well as for future online readiness orientations for themselves and their students (36.5%). In 
addition, a selection of parents reported hoping for online skills training and support for themselves 
(30.9%) as well as for their students (34.7%). Qualitative data also supports this assertion, as parents 
shared stories with a hope for further orientation and skill development for themselves and their students 
related to technology. As one parent noted, important steps they had to take included “training and 
orientation on how to use different applications, especially Microsoft Teams and Google Forms.” 
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Across the surveys, the preponderance of respondents reported using the Madrasati learning 
management system (LMS) in their schools. More than 90% of students reported using Madrasati (90.6%), 
with an 85.6% satisfaction rating. A majority of students reported using iEN TV in their learning (57.7%), 
with the vast majority of students finding it to be satisfactory (79.1%). Parent findings related to Madrasati 
and iEN TV were similar to the students’ results, with a larger majority of parents reporting their children 
using Madrasati (88.1% usage; 86.0% satisfaction), and a majority using iEN TV (54.5% usage; 80.1% 
satisfaction). Across the data, both students and parents reported high levels of satisfaction with the 
resources available to them, which enabled high quality learning experiences. 


With technology, the learning process becomes better 
and easier. The students were independent when 
studying and answering their assignments, and their 
cognitive and technological skills grew. 

— Ibrahim Haddadi, Parent 


Student data revealed a high level of satisfaction with the educational technologies used in their classes 
(88.8%), as well as with the technology and support resources available to them (79.7%). Further, students 
reported strong agreement with the notion that teachers used technology effectively in their classes 
(84.2%). Related to this, parents reported strong satisfaction with the educational technologies available 
to help their children learn online (83.3%), as well as with the way that teachers used technology (85.0%) 
in their children’s’ classes. 


DYiaatzyaksiceyal 
Summary by 
Stakeholder 


Technical staff are provided to support the learning environment and 
the technology needs of teachers. 


Agree and Strongly 
Agree Responses 


Administrators 
Teachers 


OLE Reeser 


Dimension 
Summary by Available educational technologies helped online learning. 
Stakeholder Students 87.19% 
| 
Agree and Parents — | 
Strongly Agree 
Responses | have technical and academic support when | need it. 


=A Students 79.08% 
CG ) | Parents 78.31% 


Student Support 


Related to student support, findings from the survey revealed high levels of satisfaction and agreement 
with statements affirming student support throughout the online learning transition. While students, 
parents, teachers, and principals felt the experience was comparable to in-class learning experiences prior 
to COVID-19, all stakeholder groups believed that moving to an online learning environment improved 
students’ digital literacy and that classes were run more efficiently by the teachers after moving to an 
online learning environment 


Based on the data displayed in 

the table below, majorities of 
administrators and teachers | 
believed that students had 

access to the appropriate cS 
support personal to address ee 
technical and academic arn 5) > SU | 
questions. As the numbers d-Yelalale) (etcalar in | felaall ul evelae 
indicated, administrators a ate peaks py Nay 

reported this at a higher rate, 

but teachers still agreed to this 

effect at nearly a two-thirds rate. The most significant finding, however, related to student support, 
emerged from the reported improvement in students’ digital literacy skills thanks to the support provided 
to them. Both teachers and administrators agreed with statements to this effect at rates above 93%. 


Moving to an online learning environment during COVID-19 improved the 
students’ digital literacies skills (i.e., use of digital tools, work in digital learning 


environments) 
- Teachers & Administrators (Over 93% Agreement) 
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Students have access to appropriate support personnel to address 


Imens student technical and academic questions, problem reporting, and 
complaints. 


Administrators 81.66% 
66.59% 
Moving to an online learning environment during COVID-19 improved 


> : aes the students’ digital literacies skills (i.e., use of digital tools, work in 
a << digital learning environments) 


Administrators 93.89% 
93.60% 


Continuing related to student support, teachers reported that students were supported through the 
provision or orientations, information and resources for academic and technology support. 
Administrators reported high rates of agreement with the statement that students have technical and 
academic support when needed. These results are shown in the table below. 


Students are provided orientations to their online classes, the class 
design, and the technologies used in the class. 


70.4% 


Students are provided information on technology and academic 
support resources. 


[e 72.8% 


Responses My students have technical and academic support when they need it. 
11.5% 


Policies, processes, and resources support students with disabilities. 
69.0% Administrators 


Students and Parents reported a strong majority of agreement related to access 
to support personnel to address technical and class-related questions, as well as 


to technical and academic support when needed. 
- Teachers & Administrators (Over 93% Agreement) 
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Findings for student support from students and parents revealed similar patterns of satisfaction and 
agreement with respect to the provided support during COVID-19. Both students and parents expressed a 
sense of being supported through orientations to their online classes, as well as with resources related to 
technology and learning. Both groups also reported a strong majority of agreement related to access to 
support personnel to address technical and class-related questions, as well as to technical and academic 
support when needed. These results are displayed in the table below. 


Student orientations to their online classes helped explain the class 
design, and the technologies used in the class. 


Dimension 786%) Students 
Summary by Parents 
Stakeholder 


Students are provided information on technology and learning support 
resources. 
75.5%) Students 

Agree and 
Strongly Agree 

Responses Students have access to appropriate support personnel to address 
student technical and class-related questions, problem reporting, and 
complaints. 


Parents 


68:79) Students 
Parents 

| have technical and academic support when | need it. 

79.0%) Students 
Parents 


Training Support 
Survey findings showed that one silver lining that emerged from the pandemic, insofar as it related to 
quality online instruction, was that both administrators and teachers overwhelmingly found that moving 


to an online learning environment improved teachers’ digital literacies skills (95.7% for administrators; 
94.5% for teachers). Further, more than 93% of both groups also found that moving to an online 
environment improved the institutional culture related to digital learning (93.4% for administrators; 
93.0% for teachers). What these findings suggest is that while the pandemic has been devastating and 
costly in many ways, educators in the Kingdom of Saudi Arabia did persevere and innovate through the 
difficult challenges COVID-19 presented. 
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Moving to an online learning environment during COVID-19 improved 


Dimension the teachers’ digital literacies skills (i.e., use of digital tools, work in 
STTanlanklavmeam digital learning environments) 


Stakeholder 


PN Acc\--l ale! 
Stee) aye \-veag-\-muus Moving to an online learning environment during COVID-19 improved 
Responses the institutional culture around digital learning. 


Teachers 


Teachers 


With such a large ramp-up in online learning happening in such an expedient way, technology supports 
were a critical necessity. Based on the survey data, more than three-quarters of teachers responded with 
agreement that they had access to support for educational technologies when needed (75.1%). Nearly 
two-thirds of teachers also reported having support related to instructional design and media (64.9%). 
Beyond training and support, teachers also reported a sense of satisfaction with the level of security for 
online learning (83.2%). 


Moving to an online learning environment during COVID-19 improved 
my digital literacies skills (i.e., use of digital tools, work in digital 
learning environments) 


DY iaatacieya 
Summary by 
Stakeholder 


Students 
Parents 


84.46% 


Students are provided information on technology and learning support 
resources. 


75.55%) Students 


Agree and 
Parents 


Strongly Agree 
Responses Students have access to appropriate support personnel to address 


student technical and class-related questions, problem reporting, and 
complaints. 


68.74%, Students 
Parents 


COL [ad beeen 
Evaluation & Continuous Improvement 


DYratatitela If given the opportunity, | look forward to offering fully-online 
instruction at my school again. 


Summary by 80.8% Administrators 
Stakeholder 631596) Teachers 


Online classes have comparable outcomes to onsite classes. 
52.7% Administrators 
Agree and 53.9%) Teachers 


Strongly Agree Learning that takes place online is comparable to learning that takes 
Responses place in the onsite class. 


ay 52.9% Administrators 
@ 534%) Teachers 


Related to monitoring and 
engaging in continuous evaluation 
and reflection for improvement, 


teachers shared a number of “| liked the strong connection between 
sae ears en 
revealing insights these findings MAS aMealeelarenuncece alee Msvcielecaiekslle 


are displayed in the table below. 
While a slight majority ofteachers [i ofe) ate m=». (kine mmm MalgelUl-aame)aleal-m(-r-]aallat eam 
shared that students’ grades were 
similar across class modes 
(53.2%), more than two-thirds of 
teachers responded that students 


became more aware of the students’ status, 
and the families are now involved in the 
tended to be as satisfied with their HMM (2-] dal latem 9) colacsiom-] are Ma al=\ Aci ale) Var] 9) 9) ¢-Lelf-lateye 


online class experience as their 
onsite class experience. A large 
majority of teachers reported 
collecting feedback to improve 
their classes, either individually or 
at the school level. Finally, less 
than a quarter of teachers surveyed reported a recommendation that their school avoid 

expanding online classes beyond the pandemic. Thus, the results for evaluation and continuous 
improvement suggest a body of reflective practitioners who saw collected data and reported satisfied 
students are generally not opposed to the expansion of online learning. 


io -Xol aloe) i=) ale ia K=y-\ 0 1 =) «ae 
- Faisal Alyazyidy, School Leader 
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Grades achieved are similar across class modes (online and onsite). 
Teachers 


Dimension ee ee , 
Students tend to be as satisfied with their online class experience as 
Summary by their onsite class experience. 


aW-¥-(elal-1 65 Teachers 


| collect student feedback to improve my classes. 
Agree and 


Strongly Agree 
Responses 


Teachers 
My school collects my feedback to improve support for quality online 
education. 
Teachers 

| would recommend that my school avoid expanding online classes. 
Teachers 


The findings from students and parents, related to evaluation and improvement of online learning, also 
demonstrate favorable findings for the students’ experiences. As the table below shows, more than two- 
thirds of students reported that online classes have comparable outcomes to onsite classes (67.6%), while 
just under 60% of parents did. Similarly, a majority of both parents and students found that online 
learning was comparable to onsite learning. 


WW Kol gem uar-]aman comidalice Me) Moi (o(-Jal aswel ole) an-vemant-lme)alliar-melt-ki\-m ar Ni 


comparable outcomes to onsite (67.6%), while just under 60% of parents did. 


Perhaps most importantly, more than 80% of students (80.1%) and more than three-quarters of parents 
(75.8%) supported recommending their school strategically expand online learning in the future. 


; ; | would recommend that my school strategically expand online learning 
DY Taat=yarsiceyal in the future. 


Summary by 80:15%) Students 

Stakeholder 75.88% Parents 

Online classes have comparable outcomes to onsite classes. 
Agree and 67.76%) Students 

Strongly Agree 59.37% | Parents 


Responses Learning that takes place online is comparable to learning that takes 
place in the onsite class. 


65.9799 Students 
Parents 
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Recommendations 


To ensure quality in online education, the statistical findings and expert review of the survey 

have informed a series of propositions to guide decisions and resources for future strategic 

planning. 

1. Ensure students have access to a laptop, Internet, and a supportive study environment. 

2. Provide strategies for skill building for students and parents in time management, organization, 
and technology work productivity applications through an online learning readiness intervention. 


3. Implement professional development for online teaching with teacher incentives to improve use 


of time and engagement of students. 


While it is 2021, the pandemic has reminded the world that there is still a digital divide that became more 
obvious when entire countries transitioned online to emergency remote instruction. The top challenges 
for students, parents, teachers, and administrators are unquestionably Internet costs and technology 
(hardware and software) costs. 


Students’ and parents’ top 
suggestions for future investment is 
in student broadband Internet and 
student laptops. Principals and 
teachers also noted access to a good 
work environment, skills in using 
technology, and technical support 
services with needed investments in 
student broadband Internet, student 
laptops, and technological 
infrastructure (e.g., core learning 
technologies). While it is 2021, the 
pandemic has reminded the world 


that there is still a digital divide that became more obvious when entire countries transitioned online to 
emergency remote instruction. The top challenges for students, parents, teachers, and administrators are 
unquestionably Internet costs and technology (hardware and software) costs. 
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Although there were high levels of satisfaction for all factors, the lowest area for students was technology 
support and resources. Teachers and parents, although above average levels of agreement, also report 
slightly lower levels for technical support. Several parents also noted through the qualitative data a need 
for further orientation and trainings, both for themselves and their students. This was particularly true for 
the specific technology platforms and applications that students needed to submit and complete 
assessments. 


Principals also reported lower levels of agreement with statements regarding availability of technical staff 
to support the learning environment and the technology needs of teachers. While principals felt that there 
are adequate policies and procedures to support online learning, developing new policies and providing 
resources to ensure technology access, including laptops and Internet access, as well as appropriate 
support are critical to future strategic planning to ensure quality. 


aS Recommendation 2: Time Management and 


Online Learning Readiness 


The flexibility of online education alters the pacing of learning and requires new thinking around the use 
of time, synchronous or real time and asynchronous or over time. While often online learning may 
replicate onsite or face-to-face learning with extensive online live or synchronous meetings, this can be a 
challenge for students working at a distance and parents supporting them in the home. The top 
challenges for students and parents include time management and length of time spent online. Principals 
report time spent online as a top challenge as well. Top endorsements from students and parents include 
an investment in supporting student time management and organization skill development. 


Students’ ability to manage time is a well-noted needed skill for success in online learning and should be 
built into student online orientations and online readiness interventions. While there was strong 
satisfaction with all factors, parents rate satisfaction with the orientation to online learning lower. 
Parents, teachers, and principals recommended investments in orientations to online learning. 
Incorporation of time management strategies could improve the perception of the effectiveness and 
satisfaction with orientations to online learning. Also, professional development for teachers in designing 
with the flexibility of time and using live, synchronous, wisely in their teaching and incorporating 
asynchronous activities or activities that can be done outside of a live class meeting independently or in 
groups is critical. This will be discussed in the next recommendation. 
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In addition to improvement of orientations to online, the findings 
suggest a need for an online learning readiness intervention to help 
children and parents in skill development to learn online. Parents, 
teachers, and principals recommended investments in student and 
parent online readiness and student online work skills training and 
support (e.g., Microsoft Word, Google Docs). Opportunities for parents 
and students to learn about effective practices in time management, 
organization, technology applications, and other skills needed to 
learn effectively online can increase satisfaction in the online 
education experience and learning effectiveness. Moreover, parents 
and children can learn about learning online and metacognition in an 
online learning readiness intervention. 


Disseminating effective practices in learning online through online readiness intervention, training, and 
assessment can help improve confidence in students and parents in learning online and supporting their 
child’s learning online. Distinctly, an online learning readiness intervention can address issues with 
technology access, technology skills development (using applications, communicating via technology), 
and learning skills development (organization, time management, communication, and others). These 
interventions will improve students' digital literacy skills. 


nccxeco)anvaat=aler-)i(e)ameem@)allial-im K-y-Lelallatem ea ge)icsylelar-) 
s_] 


DY=\V{=1 (eo) o)aat=lais 


Teachers report that a top area investment should be made 


in teacher professional development training and teacher Vv Investin Teacher professional 
incentives. Teacher professional development for online Development 

and incentives is proven to improve learning and : ; 

instructional effectiveness. As described in the previous VY Provide Online Teacher 


eer ae orn there is the potential to improve Professional Development 
e learning experience by including opportunities for 

parents and students to learn more about learning online to Opportunities 

ready themselves for future experiences, there is also the 
potential to improve teaching and instructional 


¥ Improve orientation for online 


effectiveness through online teaching professional learning and technical 
development. Principals report that curriculum, class, and 


instruction are designed and updated based on data, support. 
evidence, and standards. These findings provide an 
opportunity for such improvement. 
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There is an opportunity for online teaching professional development for teachers to learn how to use 
time more wisely in courses with a variety of synchronous and asynchronous teaching strategies. As online 
offers flexibility, there is /ess of a need to replicate the live and real-time environment of a traditional 
onsite class and more opportunities when activities are developed over time at a students’ own pace 
individually or with their peers. 

Although teachers agree that there was sufficient training and orientation for online learning and 
technical support, it was rated lower than other areas. Administrators align in that their levels of 
satisfaction are lower for teacher support, support staff for supporting design, technology, and media 
development, and training and orientation to online learning than other areas although still generally 
satisfied. Therefore, providing 
teachers with additional 
orientation and training for 
online teaching could better 
prepare them and increase 
their satisfaction with their 
online teaching experience. 


In addition to time 

management, student 
collaboration and engagement 
could be improved. Teachers 
report that although they are 
generally satisfied with 
engagement, it is lower than 
other areas. Teachers also report 
collaborations among students lower, while students are looking for investment in more interactive 
activities, such as games and simulations, that are more engaging than traditional onsite teaching 
approaches moved online. Often in schools, when learning is moved online the student fun and 
excitement of interacting with their peers is lost. Through professional development teachers can work 
with experts and each other to design and integrate more of these opportunities into courses. 


Finally, parents may be seeking additional communication from teachers. Sometimes parents are 
supporting students, tutoring them, and helping them manage their responsibilities, yet are unclear 
themselves as to what is expected, deadlines, and requirements. Although onsite teaching usually focuses 
on the students, professional development can help teachers establish mechanisms in their courses to 
include parents in the communication planning and distribution so that everyone can work together to 
enhance student learning online. 
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£4} Recommendation 4: Update Policy, 


@ Regulations, and Standards 


In support of the accomplishments made and response to an unprecedented global pandemic rapidly 
expanding the use of distance learning modalities, a final recommendation with respect to establishing a 
post-pandemic normal-state and global reality is to review the policy, regulatory, and quality standards 
framework used at present as this has been undoubtedly influenced by the proliferation of digital learning 
across the Kingdom in the past year and a half. Such a review will allow the Kingdom to pilot and integrate 
emerging technologies, pedagogies, and methods for assessment that will see it continue to lead on the 
international playing field of online learning. 


The KSA prides itself on a comprehensive and robust policy and regulatory framework to support and 
ensure learner competitiveness, educational quality, and focused student success outcomes. The 
accomplishments of the framework have included quality administration and staff, strategic planning and 
continuous improvement cycles, and marked investments in technology systems and access which aim to 
minimize and eliminate a digital divide across the nation. As the Kingdom of Saudi Arabia continues to 
emerge from the COVID-19 pandemic and transition toward more innovative, resilient, and continuously 
improving systems of education, the policies and standards guiding and shaping quality online learning 
will need refinement and enhancement based on these essential lessons learned. Dimensions and 
standards for online quality have surfaced in importance and in awareness in ways that would have taken 
years to rise to the top had the volume and scale of digital learning not have taken place since 2020. 


To ensure the continued success and development of online learning across K-12 education across the 
country, updates to the policies and standards that shape online learning should be reviewed and, if 
necessary, updated, and the regular or periodic basis for continued updates be refined relative to the level 
and amount of online and distance learning adopted in the post-pandemic world. Such regular reviews 
and updates will ensure a quality focus going forward for students and their families, as well as for the 
teacher and administrator stakeholders at present registering support for online quality. 


Such reviews will allow the Kingdom to pilot 
and integrate emerging technologies, 
pedagogies, and methods for assessment 


that will see it continue to lead on the 
Talkedaar-1eco)ar-lme)ty’ilay-milsi(e Moy meyal liam (=t-]aallalea 


FOR C] oss eeyys 
Learning Management System 
(LMS) Study Framework and 


Context 


i About the Madrasati Learning Platform 


The Madrasati learning management system is an interactive platform launched by KSA’s Ministry of 
Education in August 2020 and is supervised by the General Department of E-learning and Distance 
Education. While a previous iteration of the platform has been in use since 2014, the COVID-19 pandemic 
accelerated the Ministry of Education’s implementation of Madrasati at a national scale. The platform is 
accessible for the KSA's education stakeholders through individual accounts for students, teachers, 
school leaders, and educational supervisors, including all of the country’s public and private schools. 


Madrasati provides robust delivery of e-learning, similar to many options more often seen by a large 
virtual or online school provider at a district or organization. 


Madrasati Usage Across KSA and Early Success 


Madrasati is a fully integrated LMS that supports both synchronous and asynchronous e-learning options 
for students in KSA. For students who use the platform, Madrasati offers a portal to their school schedules 
and courses, including learning materials and content like live and recorded lectures and slide 
presentations, assessments, activities, tools for student-to-teacher and student-to-student 
communication, a dashboard and reporting on their progress, and a calendar. Madrasati integrates 
Microsoft Teams to support synchronous virtual class sessions with teachers as well. 


The rollout of the Madrasati platform has been nothing short of remarkable across the country, launching 
at the beginning of the 2020 fall semester in late August. Since its launch, the swift adoption by 
stakeholders across the country, most notably from students and teachers, suggests a robust success. By 
the end of October 2020, the eighth week of the semester, nearly 5 million public and private school 
students had accessed the platform, amassing some 229 million visits. By the end of November 2020, the 
Madrasati platform had seen a reported 6 million students - approximately 99% of the total K-12 
population - access the platform and had amassed 292 million visits. This includes 63 million synchronous 
virtual classroom sessions delivered, with more than 1.2M class sessions conducted daily. More than 

13 million assignments and quizzes completed (Saudi Ministry of Education, 2020). 
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Madrasati User Response Data 


In the fall of 2020, the Online Learning Consortium published stakeholder and user feedback on Madrasati. 
In a widely distributed, national survey that included more than 189,000 respondents—including students, 
parents, teachers, principals and supervisors—the overwhelming majority of stakeholders reported 
satisfaction with the platform. 


Teachers Principals Supervisors Students Parents 
Overall 
Satisfaction 
Source: Online Learning Consortium. (2020). The State of Online Learning in the Kingdom of Saudi Arabia. 


90.6% 93.9% 88.2% 86.3% 87.0% 


Below, specific survey* items are displayed for each stakeholder and user group. 


+ 196K + 4 


Number of completed surveys Stakeholder groups 


Partcipants 


m Supervisors 
m Students 

@ Teachers 

m@ Parents 


@ Principal 


Student satisfaction 
e 85.0% reported ease with logging in to the virtual classes through Microsoft Teams 


e 82.1% reported ease in interact with their teachers 

e 79.3% reported ease in accessing their schedule and class preparations 
e 78.8% reported ease in interacting with their peers 

e 78.7% reported ease in accessing and completing exams 

e 76.4% reported ease in completing the required assignments 
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75.2% reported ease in accessing the required assignments 
72.7% reported ease in receiving feedback on my performance from their teachers 
71.9 reported ease in receiving assessments and grades from their teachers 


Teacher satisfaction 


94.9% reported ease in attending virtual classes and delivering lessons 

94.1% reported ease in accessing the school schedule and creating a virtual classroom 

89.5% reported ease in interacting with their students 

86.3% reported ease in recording students’ attendance 

84.6% reported ease in attending virtual meetings with the school staff 

83.2% reported ease in adding extra curriculum, assignments and lessons 

81.0% reported ease in building and scoring exams 

75.8% reported ease in viewing students' answers to assignments 

73.3% reported ease in giving students the chance to view their performance on assignments 
72.2% reported ease in assessing students’ performance on assignments 


Parent satisfaction 


76.3% reported ease in communicating with their children's teachers 
74.6% reported ease in communicating with the school staff regarding their children 
72% reported ease in following their children's achievement reports 


Principal satisfaction 


85.4% reported ease in verifying data matching for students, teachers and classes 

80.9% reported ease in setting the school schedule 

79.1% reported ease in following up with the teachers and school staff to accomplish their tasks 
78.1% reported ease in assigning different roles and tasks to the school's staff 

76.2% reported ease in holding virtual meetings and communicate with the school staff 

66.8% reported ease in posting school announcements 

66.4% reported ease in accessing school reports to make the right decisions 


Supervisor satisfaction 


84.2% reported ease in attending teachers’ virtual classes 
83.0% reported ease in supervising teachers 

79.5% reported ease in evaluating teachers’ performance 
74.0% reported ease in holding virtual meetings with teachers 
65.8% reported ease in providing feedback to teachers 


This response data shows that across user groups’ individual items, nearly all had an efficacy and ease 
response (“very easy” or “easy”) above 70%. Based on these results, the early indications are that the 
Madrasati platform has been a unique success for stakeholders at all levels of the education system. 
Students and teachers both responded with a sense of efficacy and ease for engaging in the learning and 
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teaching process, as well as completing assignments. Principals reported confidence and ease in using the 
platform to manage the operations of schools. Parents reported a sense of ease in supporting their 
students and communicating with the school, and supervisors shared that they could effectively support, 
observe, and evaluate teachers. 


The International LMS Landscape for K-12 
Education 


Global LMS Implementation: An overview 


A learning management system (LMS) is an online platform that enables the delivery and transaction of 
communications, learning materials, resources, tools, and assignments between educators and students, 
in both physical and digital classroom environments. Notably, the LMS allows teachers to offer tailored 
instruction that can be accessed by students anytime, anywhere without geographic constraints. At its 
best, LMS implementation can create a simpler and more efficient teaching and learning experience for 
students, teachers, administrators, and even parents. An effective LMS also offers features that improve 
student outcomes and supports the scaling of a viable and guaranteed curriculum for all students, while 
promoting teacher efficacy. Essentially, the LMS enables a “one stop shop” for multiple stakeholders 
throughout an education system to access and contribute to a learner’s success. 


Broadly, but not wholly, decisions to adopt LMS in K-12 education have been often made at a local level 
rather than at a state or national level. Historically, education ministries and governments across the 
international community tended to formally adopt LMS less frequently at the national levels, often as a 
measure of cost savings and practical implementation. One tendency within this history has been to use 
options like localized servers and platforms, rather than widely distributed, cloud-based strategies. 


At the national level, learning 
resources in many countries have 
been based more ona distribution 
model. In this model, ministries and 
departments of education may 
create and/or curate a website(s) 
with resource content, learning 
materials and digital educational 
resources, rather than support a 
formal, interactive LMS. These 
resources often include rich content, 
lesson plans, and assessments, as 
well as links to other broadly 
distributed platforms or content 
providers. During the pandemic, 
many governments have also been able to secure licenses to commercial or institutional content. This 
model is often more feasible at a national government level than trying to establish the detailed logistics 
necessary to develop and implement an internally-developed learning management system, which might 
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require intensive coding and development, as well as planning for interoperability with complementary 
student information systems and other data support systems. Adopting commercially developed LMS 
platforms is eas/er, as most come with out-of-the box support to align with other information systems but 
requires a greater level of established infrastructure and support capability than hosted or shared 
curriculum and resources via the distribution model. 


One reason for this is likely related to the amount of financial and organizational capital and maintenance 
required to host, operate, and maintain such systems. For example, consider the challenge of managing 
individual student and user accounts, which are a key function of many LMSs. The challenges of accessing 
such systems are both financial and technical, rendering most solutions beyond text-based 
communication at low cost a challenge in many locations. Ultimately, for those that can adopt a formal 
internally developed or purchased LMS solution, the commercial cloud-based approach (rather than local 
institutions hosting) is a more viable approach, for the aforementioned reasons. 


Adoption during COVID-19 


Due to the emergent COVID-19 pandemic throughout 2020, many nations have rapidly adopted e-learning 
solutions and platforms as a rapid response to wide school closures and the emergence of remote and/or 
at-home learning for millions of students across the globe. Early March saw more than 1.2 billion students 
across the globe transition to remote or at-home learning (Li & Lalani, 2020). This disruption has 
accelerated institutional and organizational adoption of LMSs, as well as prompted educational 
authorities to reconsider how they provide the groundwork or necessary environment or prerequisites for 
e-learning. 


Challenges to LMS Implementation During COVID-19 

While some nations have been able to effectively contract with commercial LMS providers or develop an 
LMS internally, many nations - and indeed entire global regions - have not been able to develop 
nationwide platforms. This is may be due to a lack of infrastructure for and/or access to stable, high-speed 
internet connectivity, student and teacher access to devices, teacher and school access to digital teaching 
materials, or quality systems for delivery, assessment, and measurement at a local or regional/national 
level (Belay, 2020; Huang et al., 2020; World Bank, 2020). These challenges can be particularly exacerbated 
in rural areas of many countries, where governments and ministries may not have the capacity to support 
rural educators and learners. 


In one case study of a strained national response to 
COVID-19, Belay (2020) reviewed Ethiopia’s challenges to 
meet the needs of students suddenly forced out of 
school in the spring of 2020. Most notably, the broad 
delivery of e-learning solutions via LMS to K-12 students 
in Ethiopia was largely infeasible, due to a lack of 
essential technological infrastructure and delivery 
capability. For example, ubiquitous access to high-speed 
internet is not available nationwide, as only 12% of rural 
areas have access to the Ethiopia’s grid electricity (World 
Bank, 2019). As such, delivery of e-learning through a 
LMS platform in a country where there are more mobile 
phone connections than home internet connections might be considered a secondary or tertiary need for 


a wwas 


55 


ONLINE LEARNING 
CONSORTIUM 


educational support. While Ethiopia responded by providing educational content via radio and television, 
Belay noted other challenges and disparities, including broad socioeconomic disparities, the lack of 
quality of school resource support and quality in rural areas, and the ability of teachers and parents to 
support newly remote learners. While a single snapshot of one nation’s challenges, these difficulties are 
not unique to Ethiopia; rather, they are representative of many national responses where establishing or 
provided e-learning coursework through an LMS was not feasible. This example shares similar sets of 
conditions and constraints with the responses in other African nations, including Ghana, Zambia and 


In the Eastern Caribbean region, the Organization of Eastern Caribbean States (OECS) reported on the 
member states’ efforts to strategically plan and launch their response to COVID-19, which included 
transitioning to digital education systems (OECS, 2020). In the report, the OECS noted that while most 
every country was able to implement some kind of digital learning strategy, all of them experienced 
challenges with a lack of ubiquitous internet connectivity and device availability for students. Across the 
region, states were more likely to adopt the organization and distribution of physical materials for parent 
pickup, or delivery of instruction through mobile-based apps like WhatsApp (OECS, 2020). As of March 
2020, the members states of the OECS were working with telecommunications companies and 
philanthropic organizations to expand affordable internet service and device provision across the region, 
in order to create the infrastructure for expanded e-learning capabilities. 


LMS Implementation in North America 


Like much of the rest of the world, North American countries have responded quickly to the COVID-19 
pandemic. Those that implemented LMS-specific solutions are highlighted below. 


The Response in the US and Canada 

In the United States (US) and Canada, the pandemic has accelerated the formal adoption of LMS platforms 
for K-12 institutions, local education associations (LEAs) and regional and national departments and 
ministries. As Hill (2020) has reported, an estimated 60% of K-12 LEAs in the US and Canada that enroll 
more than 2,000 students - a staggering 85% of total enrollment in these countries use at least one LMS. 
Further, approximately 80% of students are enrolled in a district which has formally adopted an official 
LMS. Hill also notes that this year has been the first time that many schools and districts have agreed to 
use a LEA-wide LMS solution. 


In the US, in particular, several state-level departments and boards have entered into formal agreements 
with LMS providers since the beginning of the pandemic. Most states have language in their 2020 
contingency planning for COVID-19 that strongly recommends that districts and schools in their state 
adopt an LMS, if they haven’t already. For example, in Massachusetts, guidance from July 2020 (and 
updated in October 2020) by the Massachusetts Department of Elementary and Secondary Education 
(MDESE) noted that all districts were “strongly encouraged” to implement a LMS solution, and that Google 
Classroom, Canvas, and Schoology are three of the learning management system platforms most widely 
used in the state (MDESE, 2020). 
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Figure 1: Yearly New Implementation of the LMS in K-12 Institutions (North America) 
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Figure 2: Number of US & Canadian Districts Adopting a New LMS as of End of September 2020 
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Figure 3: K-12 LMS Usage in US and Canada 


K12 LMS in US and Canada 
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US states with formally adopted LMS or master contracts for LEA purchasing 


NUT na] eX-1are) a (over-| | 


US State eae acne educational alee ol Number of students 
agencies (LEAs) 
Alabama Schoology 139 1,463 747,042 
Alaska Canvas 54 507 132,104 
Arizona Blackboard Learn 738 2,449 1,269,584 
Delaware Schoology 45 226 136,027 
Hawaii Blackboard Learn 1 292 179,698 
lowa Canvas 333 i syeul OWS 2. 
Maryland Blackboard Learn 25 1,445 879,601 
Nebraska Canvas 244 1,004 323,391 
New Jersey Maestro LMS 678 2,516 1,369,715 
New Mexico D2L 128 853 338,112 
North Carolina Canvas 115 2,705 1,490,472 
Pennsylvania Odysseyware / 782 2,971 1,719,336 
Edgenuity 
Ohio Internally developed 1,653 3,853 1,983,469 
Oregon Canvas 197 1,239 576,407 
South Carolina Canvas 102 1,268 804,158 
Texas Schoology 1,203 Samal 5,343,893 
Utah Canvas 163 1,113 652, 348 
Vermont Canvas 60 308 84,387 
West Virginia Blackboard Learn Sf 762 282,879 
Wyoming Canvas 48 B59 O8yGr 


Sources: Walbank, 2020; Lieberman, 2020; Texas Education Agency, 2020; State Educational Technology Directors 
Association (SETDA), 2020. 
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Elsewhere in North America 
Beyond the US and Canada, a variety of 
responses have emerged, mostly in line 
with a distribution model, with one 
notable exception. In response to the 
pandemic, Jamaica has announced a 
plan to develop an internally developed 
LMS for national implementation. The 
Ministry of Education, Youth and 
Information announced plans in July 
2020 to establish a national LMS, in 
partnership with undisclosed providers. 


Across much of the rest of North America, the distribution model has been adopted, largely due to 
constraints of infrastructure and lack of access for learners. For example, Mexico has launched their 
Aprende en Casa (“Learning at Home’) platform, which primarily consists of audiovisual broadcasts and 
recordings distributed via television and curated through a distribution model on the Aprende en Casa 
website. Like many countries around the world, the infrastructure challenge of limited access to internet 
and connected devices limits the extent to which learners in Mexico can leverage an LMS platform for 
more robust e-learning. 


In El Salvador and the Dominican Republic, ministries of education have set up websites that provide 
access to content and lessons for students. A recent World Bank (n.d.) report has noted that in El Salvador, 
“a call center, staffed by five people, accompanied by six subject-area specialists, responds to related 
questions around the clock.” Both countries are also leveraging social messaging applications like 
WhatsApp to deliver content. 


Finally, several countries across the Caribbean, including Belize and Barbados, have leveraged the 
resources and platforms of the Caribbean Examinations Council (CXC), which historically has supported 
postsecondary entrance testing. During COVID-19, however, the CXC has established a learning platform 
that includes learning content and virtual classroom capability. 


LMS Implementation in Asia 


LMS implementation across Asia. South Korea was one of the first countries affected by COVID-19, 
reporting its first case in January 2020. The Korean Education and Research Information Service (KERIS) 
reported in April 2020 that K-12 education across the country moved online in response to the pandemic 
(KERIS, 2020). As Cha and So (2020) noted, the Ministry of Education in South Korea offered three primary 
ways for students to engage in online learning during the shift to online learning: real-time interaction, 
contents-based online class, and assignments-based online class. The primary LMS vehicle used for the 
contents-based online class was the e-Hakseupto platform, where students could access lessons and 
content (KERIS, 2020). Educators throughout South Korea also continued to leverage the communication 
tools and platforms of other commercial providers, including Google Classroom and Canvas. 


China hosts the world’s largest K-12 student population on the planet numbering over 176 million 
learners. This makes the Chinese education system the largest state-run education system in the world 
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(Gulati, 2020). China, the epicenter of the COVID-19 outbreak, issued school closures between late January 
and early February 2020. In response, China launched a national cloud learning platform for junior and 
senior high school students, and started broadcasting primary school classes (Jee, 2020). EdSurge 
reported that two of China’s nationally supported platforms, Empower Learning and Educloud, which had 
existed prior to the COVID-19 pandemic, were augmented with the support of Chinese tech companies to 
provide service during remote learning in the early part of 2020 (Ning & Corcoran, 2020). In addition to 
these platforms, students in China also used a variety of apps and platforms, including the WeChat app, 
Alibaba’s Future School, and Tencent Meeting. These technologies supported synchronous and 
asynchronous learning, in addition to the LMS platforms. 


In India, the Ministry of Human Resources Development established a number of platforms solutions for 
distance learning across the nation. The DIKSHA and SWAYAM platforms provided online courses and 
content, while the e-PATHSHALA and other digital content repositories provided access to learning 
materials for students across the country (Indian Commission for Cooperation with UNESCO, n.d). In 
addition to these platforms, radio and television broadcasts supplemented remote learning 
opportunities. 


In many other Asian nations, the challenge of lacking nationwide internet access, and the subsequent lack 
of pre-existing distance learning platform implementation meant that many countries were pushed to 
respond to COVID-19 by beginning their gap analyses and strategic planning to launch. For example, 
Bangladesh, Pakistan, and Philippines are all countries that reported an identified need to further 
understand their gaps and opportunities to better establish distance learning platforms (Bangladesh 
Ministry of Education, Ministry of Primary and Mass Education, 2020; Ministry of Federal Education and 
Professional Training, Government of Pakistan, 2020). While most nations analyzed were able to provide 
emergency, remote distance learning support to students through television and radio broadcasts, the 
pandemic more often exacerbated the needs for robust online learning platforms than it did reveal 
nations’ strategic readiness. 
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Comparison of Madrasati to Top LMS 
Providers in K-12 Contexts 


Profile Benchmarking: Madrasati with Commercial LMS 
Providers 


After a review of the LMS landscape in global K-12 education, it is worth comparing the KSA’s effort to 
launch Madrasati across public and private K-12 schools during the global pandemic. That is because in 
just a few short months, KSA has launched an internally developed solution, capable of integrating with 
the Microsoft 365 cloud platform and Microsoft Teams, to create a learning platform that is comparable 
with the fee-based commercial services that many other national, regional, and local education 
authorities are leveraging. While most countries around the world have been forced to adapt to the rapid 
disruption of COVID-19, it is worth noting that Madrasati’s functionality compares with the options 
provided by commercially adopted systems. 
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Estimated Year of initial 
Development type 


number of users* _launch/ release 


Canvas ~30 Million : ; 
Gnetuete) Pe lanice 2011 Yes Commercial provider 
Edmodo ~88 Million ; : 
(NetDragon) worldwide 2008 NES Commercial provider 
Google Classroom ~100+ Million Conmne teial proviger tee 
8 : 2014 Yes integration w/ Google Suite 
(Google) worldwide : 
enterprise ($) 
Schoology ~20 Million : : 
(powerschaol) Noord 2009 Yes Commercial provider 
Moodle eae ten O99 Yes Commercial provider 
worldwide 
Blackboard os 
Classroom Series 2014 Yes Commercial provider 
(Blackboard) 
athe ~15 Million 2011 Yes Commercial provider 


Internally developed: Freely 


Madrasati ~6 million in KSA available to all public and 
private schools 


*Use statistics provided through provider website information, as of December 2020. Estimates include total users 
across all sectors, including higher education, private sector, and non-educational use. 


Features and Functions Desired in LMS 


Going beyond simply LMS platforms, it is 5 . : 
also worth noting that the Ministry of After review of the international 


Education’s efforts in KSA to provide landscape, Madrasati is relatively 
learning content via the iEN broadcasting peerless related to other nations’ 
channels and via YouTube also compares i 
favorably along the lines of other efforts to implementa countrywide 


distribution models. LMS for e-learning in the K-12 


education sector. That is, there are 
and associated criteria KSA has established several other platforms, but none 
for an effective LMS, followed by a table that equal the combination of scale 


Below is a framework for the components 


benchmarking the top commercial LMS Pao aa 
providers with Madrasati, related to those and sop histication of what the 


features: Kingdom has initiated. 
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Course delivery and 
administration 


Assessment and 


Design and usability WANE 


Framework for 


successful LMS 
fore) an) eXe)ar val ts) 


Course content 


Training and support management 


System 
EXelaalialsic-le(e)aie- are] 
report management 


Data management 
and security 


Course delivery and administration 


Robust capabilities for content and file organization and management 

Robust communication capabilities for synchronous and asynchronous communication between 
and across user roles 

Capability to embed/import and author rich content across content types. 

Support for Learning Tools Integrations and third-party content 

Calendar and selective release features allow for tracking of assignments 

Collaboration and social learning tools, including discussion boards, capability to create 
groups/sub-groups, and integration with file sharing tools and cloud storage solutions 

Online help and support for students. 


Course content management 


Ability to organize content in and pull from a digital repository for content across subject areas, 
including varied file formats for universal access 

Rich content and media creation, including SCORM content 

User content download/upload available for offline use 

Search functionality 
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Assessment and evaluation 

e Capability to create and grade multiple assessment types: quiz/test, file/link submissions, 
discussions. 
Quizzing/testing tool supports multiple question types 
Ability to selectively release assignments based on time, date, student ID, and other criteria 
Gradebook entry and access for students to track and gauge their own progress 
Gradebook export to student information systems (SIS) and other report 
The ability to link competencies and goals to performance and evaluation rubrics 


System administration and report management 

e Ability to archive, copy, and migrate existing courses, templates, and course files and templates 

e Ability to grant access and assign privileges to roles at systems, course, and course group levels 

e Ability to send notifications from multiple user roles across the system, including notifications 
from supervisors, administrators, and teachers. 

e Ability to create standard reports across user roles, courses/trainings, as well as qualifications and 
program completion 

e Ability to track and log students’ activity, including content engagement, assessment 
completion/submission and interactions 


Data management and security 
e Encryption of secure and sensitive data 
e Integration with student information systems, human resource management systems, and data 
management systems 
Authentication at industry standards for single sign-on (SSO) and other authentication protocols 
Regular system updating, backup, and security 


Training and support 
e Available online and/or print knowledge base, training materials, and how-to guides for multiple 
user roles 
e Available opportunities for professional learning support across user roles, via formal trainings, 
webinars, and dedicated staff support 
e Live support via phone, help ticket and support form submission 


Design and usability 
e Clear navigation and usable interface across user roles for navigating through courses, 
modules/content, notifications, and reports 
e Responsively designed to support mobile devices and agnostic with respect to device 
types/platforms 
Multiple language support 
Support for accessibility considerations, including captioning and screen readers 
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Features Benchmarking: Madrasati with Top Commercial LMS 
Providers 


Broadly, Madrasati’s features and performance largely matches that of the top commercial providers. For 
a comparison to the providers here, please see the following chart: 


Course content management 
Training and support 


Design and usability 


System administration and report 
management 


Madrasati’s Platform Tools 


Madrasati includes a set of tools and interactive media that support the creation and management of 
virtual classrooms. For synchronous delivery, Madrasati leverages Microsoft Teams for videoconference 
class sessions and engagements. For asynchronous delivery, Madrasati serves as the access point for 
content - including recorded lectures and iEN channels - assessments, discussion forums, and other 
learning activities. 


Some of the primary tools Madrasati supports for virtual learning are: 


1. School schedule: Overall school schedules, including individual learner schedules, can be 
established and communicated. 

2. Announcement and alerts: Announcements related to tasks, activities, and virtual classrooms are 
announced through the platform. 

3. Lesson preparation: Teachers can prepare lectures, presentation slides, content and media files, 
as well as set agendas for class sessions and connect curriculum to established performance 
standards. 

4. Virtual classrooms (via Microsoft Teams): Live, synchronous class sessions can be hosted through 
the platform, enabling remote/virtual instruction in real-time. 

5. Discussion forums: Both teachers and learners can communicate, discuss, and exchange 
knowledge and experiences on the platform. 
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Exams and assessments: Learners can take exams, with instantaneous grading, feedback, and 
performance analytics. Oral exams may be conducted through Microsoft Teams functionality. 
Attendance and behavior monitoring: Teachers can record daily class attendance and monitor 
and report on student behavior. 

Homework and learning activities: Student learning activities and homework assignments may be 
downloaded and uploaded to submit through the platform. 

Performance indicators: Madrasati can record users’ activities and performance in the platform, 
enabling teachers and supervisors to monitor the performance and progress of the e-Learning 
process. 

Reporting: The system provides insights into users’ performance and activity through the 
platform. 


Madrasati’s Tools and Functionality by Role/User 


Learners 
The tools and functionality available to learners on the Madrasati platform include: 


e Class preparation: School schedule and calendar 

e Learning and course engagement: Virtual classrooms for synchronous and asynchronous 
instruction, E-learning content, and learning and enrichment activities 

e E-learning content and activities: Rich content, including readings, lecture videos, and 
presentations are accessible for learners 
Assessments: Exams and assessment tools 
Communication features: Email, discussion forums, and teacher conferencing rooms 

e Reporting and evaluation: Self-evaluation, reporting, and statistics dashboard 

Teachers 
The tools and functionality available to teachers on the Madrasati platform include: 

e Course administration and management: School schedule and calendar, attendance and behavior 
management 

e Instructional design and delivery: Lesson preparation, virtual classrooms for synchronous and 
asynchronous instruction, e-learning content, and learning and enrichment activities 

e E-learning content and activity management: Rich content, including readings, lecture videos, and 
presentations can be created and uploaded by teachers 

e Assessments: Exams, exam banks, and assessment tools 

e Communication features: Email, discussion forums, and teacher conferencing rooms (to answer 
students) 

e Reporting and evaluation: Reporting and statistics dashboard 
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Administrators 
The tools and functionality available to administrators on the Madrasati platform include: 


e School administration and management: School schedule setting, grading records setting, 
content and resources management and calendar, attendance and behavior management, and 
activities and projects classification 

e Learner management: Administrators have access to student performance data and can track 
learner progress 

e Teacher management: Administrators have access to teacher performance data and can evaluate 
teacher effectiveness 

e Instructional design and delivery: Lesson preparation, virtual classrooms for synchronous and 
asynchronous instruction, e-learning content, and learning and enrichment activities 
Communication features: Email and discussion forums 
Reporting and evaluation: Reporting and statistics dashboard 

Supervisors 
The tools and functionality available to supervisors on the Madrasati platform include: 

e Enrichment activities governance: Supervisors can establish enrichment activities, and monitor 
use data and learner progress 

e Teacher evaluation: Administrators have access to teacher performance data, and can evaluate 


teacher effectiveness 
Communication features: Email and discussion forums 
Reporting and evaluation: Reporting and statistics dashboard 


Parents 
The tools and functionality available to parents on the Madrasati platform include: 


Student support: Access to school and resources and schedule and calendar 

Access to learner data: Parents can see how their learners are progressing through e-learning 
content and their performance on assessments 

Access to a variety of learning resources: Parents can access learning materials to supplement 
instruction for their children 

Communication features: Email and discussion forums 

Reporting and evaluation: Reporting and statistics dashboard 
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Benchmarking Nationally Supported LMSs 


Even prior to the COVID-19 pandemic, the international picture of LMS implementation for virtual learning 
in K-12 education has been difficult to discern. That is, the extent to which countries around the world 
have formally adopted and/or mandated the use of a specific learning management system (or systems), 
particularly prior to the COVID-19 pandemic, is difficult to determine. This is due to the rapidly changing 
nature of nations’ educational needs, capacities to plan for and implement large-scale technological 
support across their education systems, and the shifting landscape of LMS and ICT providers. 


In many nations, local or regional governments and authorities most often assume responsibility for 
selecting and implementing LMSs, as well as for providing related support and access to e-learning 
content for educators. To the extent that national governments and ministries are involved in these 
processes, these national-level entities tend to occasionally provide some funding or other support and 
possibly technical specifications and/or learning outcomes and standards. More frequently, however, LMS 
solutions are not typically mandated for use across entire countries or regions. Once a solution is adopted 
by a nation, particularly from a commercial vendor, it does typically become the default or de facto LMS in 
use, due to the resource and technical support from a centralized authority. 


Nevertheless, several reports over the past decade have shed light on how countries across the globe have 
developed e-learning solutions. Notably in 2011, the US Department of Education’s Office of Educational 
Technology conducted a study of 23 nations and their approaches to implementing ICT, including the 
adoption of LMSs. The report found that only 6 of 23 countries participating in the international study of 
LMSs had formally adopted an LMS in all or nearly all of their schools. The study also noted that 
“regardless of levels of penetration, how ministries of education, local education authorities, and schools 
conceive of and use LMSs varies greatly by country” (U.S. Department of Education, Office of Educational 
Technology, 2011). Given that nearly a decade has passed since this report, the development of LMS 
technology and progress in the facilitation and pedagogy of effective e-learning has certainly increased. 
However, much of what is publicly available and known about the international use of LMSs leaves us ina 
similar place. 


In a documentation of nation-by-nation e-learning responses to the COVID-19 pandemic, UNESCO (n.d.) 
has compiled a list of distance learning resources that many countries have made available to their 
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citizenry. Most of the resources in the list follow the distribution model, often linking to portals and 
websites that include content, rather than established or nationally supported LMSs. Some of the 
notable country supports are provided in the following section. 


Adopted LMSs 


An adopted LMS is simply an LMS deployed by an external vendor or provider. Most often, this takes the 
form of acommercial provider, such as Canvas, Blackboard, Schoology, or Google Classroom. An 
overwhelming majority of LMS deployments in K-12 institutions around the world are done through 
adopted solutions, as opposed to internally developed solutions or LMSs. Comparatively, adopted LMSs 
often have the benefit of requiring less technical expertise and effort to implement, as well as externally 
managed support and maintenance. The largest platforms also have deeper market penetration across 
education sectors, meaning that they tend to also offer robust content and training sharing. 


Internally Developed LMSs 

An internally developed LMS is one that is designed internally by an organization or school, rather than 
one that is purchased and deployed by a private firm. The origins of internally developed LMSs date back 
to some of the earliest days of e-learning itself, when many businesses and schools and universities did 
not have access to a diverse market of vendor options for LMS adoption. A recent analysis, eLearning 
Inside (2019), notes that most businesses and educational institutions have moved away from internally 
developed LMS solutions, given the costs, as well as the technical and logistical challenges of internal 
development, as well as the rich array of features that many purchased solutions provide. However, the 
development of an internally developed LMS does not necessarily suggest an inferior product. Rather, an 
internally developed LMS that aligns with the strategic goals and vision of the institution may well prove 
to be more effective, particularly given the rapid changes and growth in e-learning technology and 
pedagogy over the past decade. In this analysis, an internally developed LMS that can strategically align 
to the institution’s vision, as well as provide the rich features that most off-the-shelf providers consider 
standard to their offerings, can be an even more valuable investment for the developing entity. 


Internally developed LMS solutions are often paired with other supports, technologies, resources, and 
access points. Thus, solutions leveraging an internally developed LMS often provide students with 
supplemental benefits of a distribution system, including open resources and content repositories, 
broadcasted lessons via television and radio, and occasionally support for mobile phone credits or SIM 
cards. Use of national and public television and radio stations seems to be the primary mode of 
transmission for teaching resources and content, and largely these technologies and platforms are more 
widely available than broadband, high-quality internet access for many nations. 


Coupled with distribution model resources, internally developed LMS solutions and widely available 
access to lessons have worked to ensure that students across the globe have had continued access to 
learning opportunities during COVID-19. A list of nations that have implemented their own LMS solutions, 
along with other supports, is displayed in Appendix A. 
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» Considerations for LMS adoption 


When implementing LMS for district, state, or even nationwide adoption, there are several key factors to 
consider, including: alignment; cost; functionality and capacity; visibility; pilot implementation; user and 
technical support; customer opinion; data analysis and reporting capabilities. This section highlights the 
key components decision-makers generally need to take into consideration. 


Alignment 

One of the vital considerations for selecting an LMS - at any scale - is the alignment and fit with the 
overarching district/state/ministry goals and the extent to which the technology supports those 
institutional objectives. This alignment can take place along multiple dimensions, from pedagogical 
considerations like an emphasis on student personalization or project-based learning. The alignment can 
also take place along curricular grounds, including mandated or standardized curriculum. 


Other considerations, including the need for at-home or distance learning, or blended or hybrid learning, 
should also be aligned. Ensuring alignment with broader goals and mandates, as well as other priorities 
and initiatives, will ensure that the chosen LMS does not cascade into a string of add-ons and service 
charges for what the provider considers to be services that require surcharges and extra fees. 


Cost 

Another critical factor that organizations must consider when adopting an LMS is the cost of the platform 
and the value delivered for the initial and ongoing costs of implementation and support for the platform 
and its related services. 


Popular pricing models for LMSs often occur on a user subscription basis, incurring a regular fee per 
student, or on a licensing fee model. In addition to the ongoing cost of user access through subscription or 
licensing fees, there can also be significant costs to implementation, training, technical support, and 
maintenance. Many of these costs can be ongoing or prolonged, with some vendors wrapping these 
service costs into packages, and others offering them a la carte or on an as-needed basis. 


Functionality and capacity 

The functionality and interoperability of an LMS is a necessary consideration when implementing a new 
system. Leaders and administrators must account for the functionality they need from their LMS, 
identifying the use case needs of the various stakeholders who will use the system. For example, leaders 
must account for the needs learners will have, and should consider which functions and capabilities are 
essential or most needed. For large scale implementation, any nationwide or large-scale implementation 
will likely require robust e-learning capabilities, including course management, content and assessment 
delivery, flexible communication tools, data and reporting tools, and a high level of privacy and security. 


Ease of use 

An important consideration for the adoption of an LMS is the extent to which its users find it relatively 
accessible and usable, and the time horizon to build efficacy for use. This is particularly true in K-12 
settings, where many young students are being introduced to technological tools for the first time, and 
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when their capability to access systems requires the support of parents and/or other teachers and adults 
who can support them. During the COVID-19 pandemic, this factor has become even more critical, as 
many adults supporting learners were constrained with work or technological challenges themselves. 


Pilot implementation 

Pilot implementation is an important factor to consider for LMS adoption in many organizations, schools, 
and districts. This is primarily because the choice to adopt an LMS is such a large investment, particularly 
with respect to the complexity of integrating a new system into existing or legacy infrastructure, and to 
merging the LMS with any existing student information systems (SIS) or other reporting databases. The 
need to pilot includes and touches upon every other area of consideration listed here, from cost to 
support to privacy and security. Thus, it is paramount that any LMS adoption consider this factor. 


Support 

Support for any LMS adoption and implementation is another important factor that leaders must take into 
account. While many LMS providers offer platforms and functions that are user-centered and well- 
designed, users and stakeholders from every role across the system will need support for setting up and 
using the LMS. Administrators and technical advisors will need to liaise with LMS providers to work 
through technical requirements. Supervisors and managers will need to understand how to use the 
platform for personnel and performance indicators across both teachers and learners. Teachers will need 
the necessary and sufficient skills to use the system to deliver high-quality instruction and ensure learner 
performance management. Learners and parents will need to be able to access the system, engage in the 
course materials, complete assessments and gauge their progress. Across each of these domains of 
activity and different roles, the amount and availability of support provided to implementing 
organizations is critical to any successful LMS adoption. 


Customer opinion 

Fundamentally, any successful LMS adoption needs to meet its clients’ requirements and satisfy their 
demands for performance. Thus, customer feedback is an important aspect to consider when adopting 
LMS solutions. Further, providers should consider users’ knowledge of the institution and opinions about 
the system. 


Reporting and data analysis 

Given the demand for measuring and assessing student progress toward international and national 
education performance standards, as well as the need to provide stakeholders across the education 
system a view into schools’, teachers’, and learners’ performance, any consideration for LMS adoption 
requires robust and functional data reporting and analysis capability. In addition to reporting on student 
progress, many LMS solutions offer reporting on the use and performance of the system itself, enabling 
leaders to make better decisions about resource allocation and providing the capacity to benchmark 
against strategic priorities. Thus, reporting and data analysis is a critical adoption consideration. 
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Recommendations for Future 
LMS Innovation 


Several recent reports have noted the innovations and transformations that technology-enhanced 
learning may usher in across the education sector. CB Insights (2020), for example, noted that online 
learning increases access and invites a wide array of learners to participate in formal learning, including 
the important role LMS plays as an important component of this transformation. Further, the adoption of 
robust online learning was identified along with augmented and virtual reality (AR/VR) solutions, artificial 
intelligence (Al), gamification, and personalization in learning pathways as being potential drivers of 
innovation beyond COVID-19. 


Elsewhere, in its 2020 report on the technology enablers driving K-12 innovation, the Consortium on 
School Networking (CoSN), a leading technology leadership organization for school leaders, identified five 
key drivers that support K-12 innovation: digital collaboration platforms, tools for privacy and safety 
online, analytics and adaptive technologies, cloud infrastructure, and mobile devices. While the primary 
driver, digital collaboration platforms, comprises the LMS itself, each of the other drivers connects to and 
springs from the services next generation LMS offerings provide. Thus, an effective and broadly distributed 
LMS has the capacity to drive innovation across schools, states, and indeed, nations. While the global 
pandemic has indeed forced education institutions and ministries to react, LMS offerings and their 
associated services create opportunities to innovate beyond the current crisis. 


Future innovations in the domain of international K-12 education, including in the functionality and 
features of the Madrasati LMS, may want to consider that further incorporating emerging technologies 
and services could expand learning opportunities for students, and for their capacity building. Certainly, 
there are primary challenges to providing robust internet access to all citizens in many nations, 
particularly to those in rural and remote areas. Access to devices, supports for mobile learning, technically 
capable support staff and training resources, as well as strategic plans and policies for widespread 
adoption are all vital considerations. It is likely that nations that are able to account for and provide the 
infrastructure and support for these technologies may be able to expand on the gains in online learning 
they have seen during COVID-19. 


Within the Kingdom of Saudi Arabia, future recommendations for the expansion of quality e-learning for K- 
12 education, and specifically for the Madrasati platform include: 


Recommendation 1: Infrastructure Expansion 


e Continue to expand infrastructure and access to internet and devices Expanding infrastructure 
and access will be important across the country, and particularly so in rural areas. While not 
germane to the functionality of Madrasati LMS itself, this recommendation remains vital for KSA in 
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its efforts to create and provide quality educational experiences for all learners. Beyond internet 
connectivity and access, the ability to access the platform through computing and mobile devices 
is also necessary to ensure that learners continue to the use the platform to its fullest capabilities. 


This may include continued support for mobile users or for those who may need to access the 
platform primarily through a phone or mobile device. 


e Expand the breadth of availability and support for learning technology integration (LTI) tools. 
Similar to Madrasati’s current interoperability with Microsoft Teams, expanding and developing 
integrations with Madrasati will offer educators more flexibility and creativity with content 
creation and sourcing, and will enable students a variety of ways to demonstrate their learning 
and be creative. Furthermore, integration with other tools may allow for rich content. 


Recommendation 3: Educator Training 


e Expand support, training, and professional development for educators and administrators. As a 
pathway toward innovative e-learning beyond the remote response to COVID-19, teachers and 
administrators will need continued professional learning and support for emerging best practices 
and pedagogy. They will also need networks and peer-to-peer support systems to build 
collegiality and shared communities of practice. 


Recommendation 4: LMS Research Program 


e Develop a research program to study Madrasati’s use and effectiveness. The KSA might consider 
implementing research or continuous improvement networks to gather insights on learner, 
educator, and even parent use of the platform to improve user experiences across these roles. 


While the efficacy of cybersecurity and privacy are beyond the scope of this analysis, all ministries and 
nations should be attuned to the continual challenges and threats to data privacy and security. 


Nevertheless, with the powerful combination of the Madrasati LMS and the bundle of resources, 


broadcasted and streaming lessons, and digital textbooks, the nation is poised to lead the 
globe as a model for e-learning in K-12 education. 
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Based in part on the COVID-19 experience and recent focus of the Ministry of Education around inclusive 
digital learning programming, instruction, and tools, the above framework is a useful organizing construct 
for the work planned and being pursued at present to ensure unparalleled student achievement for all 
learners as well as to embody a continuous improvement spirit necessary for 21% century education. The 
NELC’s K-12 Online Learning Framework centers student achievement as the foundational aspiration and 
outcome of K-12 education in the Kingdom of Saudi Arabia. In this way, each strategic dimension and 
supporting initiative and implementation should make a contribution toward helping students in the 
Kingdom reach their fullest potential and is undertaken with a goal and realization of heightened student 
achievement at the core. 
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Extending from this framework are the core strategic dimensions of this framework: Leadership, Teaching 
& Learning, and Continuous Improvement. Each dimension extends to supporting sub-dimensions and 
their associated activities. These core dimensions serve as the conceptual pillars for future strategic 
direction in the Kingdom’s efforts to modernize, expand access, and deepen support for teachers and 
leaners as continued opportunities arise to maximize learning in digital spheres. 


Leadership 


Research has shown that effective leadership in schools matters and is connected to student achievement 
(Bryk et al., 2010; Leithwood & Jantzi, 2008; Orphanos & Orr, 2014). This framework dimension accounts 
for leadership—both positional and behavioral elements—that are necessary to drive innovation and 
future student achievement in Kingdom of Saudi Arabia. Leadership subdimensions include Policy, 
Technology, and Training. Each of these subdimensions and their related strategic supports are described 
in the following sections. 


Policy 

Policy accounts for the administrative and regulatory supports and guidelines that govern the operations 
and delivery of educational services. Across the leadership dimension, strong policies emphasizing and 
supporting student achievement are critical to the overall success of K-12 educational outcomes. In 
particular, policies that expand digital and online learning, whether in fully online modalities or as part of 
a blended strategy, should be expanded. 


Strategic supports for the Policy subdimension are: 
e Aligned Planning: Clear strategic alignment, both vertically and horizontally, is necessary for 


effective policy development and implementation. Vertically, the policies and expectations should 
align vertically from an overarching vision and mission from the Ministry of Education, and be seen 
on the ground level of schools. Horizontally, each level of education administration should have 
common understandings and planning supports for their unique role, value, and contribution to 
the broader policy objectives. 

e Agile Governance: In a world of rapid change and increasing complexity, effective and responsive 
governance is vitally important. As such, the structures and processes through which ministries 
and governing bodies administer education should include a design to share the authority and 
capacity to develop policies and decision-making that affect the entire educational ecosystem. 

e Innovation Hubs: As Cook (2019) has described, complex and fast-paced systems can be difficult 
places to innovate. As such, “zones of exemption” at the margins of such systems provide rich 
spaces to experiment and capture learning for innovations that might begin to filter back in, or 
even subsume, components of the existing system. In this way, the establishment of “innovation 
hubs” where administrators and practitioners claim the permission or freedom to negotiate 
policies and guidelines in service of innovative pilot programs and implementations can support 
continuous improvement in policy. It is recommended that these hubs operate within continuous 
improvement or action research frameworks, so that innovations can be captured, improved 
upon, and disseminated system-wide. 
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Technology 

While technological capacity and capabilities are essential for enabling online learning at a fundamental 
level, dedicated attention to technology as a core element of the Leadership dimension extends beyond 
the mere functional capacity to deliver online learning. 


Strategic supports for the Technology subdimension are: 


Anytime, Anywhere Access: The expansion of online and digital learning in K-12 education creates 
the possibility for new and innovative delivery and engagement modalities. The mantra of 
anytime, anywhere means that both educators and learners can creatively think about what it 
means to go to school, and for what progression through a course or school year looks like. For 
students, this creates flexibility to continually review lessons and pace themselves appropriately. 
While for teachers, this creates opportunities to provide more personalization at scale. 
Infrastructure & Device Support: As previously noted in the recommendations section, the 
capability for students and educators to be online and connected to educational opportunities is 
perhaps the most fundamental consideration for enabling online learning. In this way, students 
and educators must have robust and consistent access to the Internet, as well as the devices to 
participate in educational services. Ensuring ubiquitous and reliable Internet, as well as creating 
ways to secure devices and/or making digital learning environments friendly for many device 
types, should be considered critical infrastructure priorities in education. 


Training 

A final but important component of the Leadership dimension is the need for robust training—both inside 
of the K-12 education system with respect to professional development. Strategic supports for the 
Training dimension include: 


78 


Modernized Workforce: A key element in developing training for the Leadership dimension will be 
to transform teacher preparation programs in order to better develop teacher knowledge and skill 
in areas which align to the student achievement foci across the other dimensions and 
subdimensions. This could include the redesign of teacher licensing and credentialing, so that 
teacher competencies are aligned with learner-centered and competency-based practices that 
demonstrate mastery. Create incentives for teachers to earn the credentials necessary for 
teaching dual or concurrent enrollment courses. This also includes a need to diversify the teacher 
workforce, as well as to invest in both technology competencies and cultural competencies for 
educators. 


Long-life Learning: As Dede and Richards (2020) have offered, the notion of /ong-life learning 
suggests that that adults will need to continue to explore new skills and gain new knowledge long 
after K-12, or even higher education experiences. In this way, the inclusion of long-life learning as 
an emphasis of the training subdimension demonstrates the need to continually provide those in 
the K-12 education sector with the opportunities to continue to learn and grow through their 
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practice, whether they are a teacher, administrator, or other support personnel. Just as the 
students of today will need to continue to learn throughout their lives, educators and those 
working to support education in the Kingdom must also have access to formal and informal 
learning opportunities throughout their careers. 


e Empower Stakeholders: Across roles within both K-12 education sectors, it is critical that students, 
educators, and parents all have the knowledge and efficacy to use digital tools, devices, and 
applications. To this end, investment in both formal and informal learning opportunities, and the 
development of communities of practice that can support and extend learning is vital. 


Teaching & Learning 


Teaching and learning are the core functions that take place in educational settings, and as education 
transforms and shifts with advances in technology and society, the importance of teaching and learning 
have never been more essential. However, the ways in which teaching and learning contribute to student 
achievement must innovate and continue to improve, particularly with increased connectivity and 
capability to integrate new modalities of learning and dynamic learning environments that both reinforce 
and extend beyond students’ experiences in the classroom. Teaching and Learning subdimensions 
include Design, Engagement, and Assessment. Each of these subdimensions and their related strategic 
supports are described in the following sections. 


Design 

As Richter and Allert (2017) have noted, school is itself a product of design, of artificiality that is not 
immaterial but rather contingent upon the socioeconomic contexts in which it functions. Thus, the ways 
in which educational experiences are designed may be a matter of continual interrogation, reimagination, 
and revision. For this subdimension, three strategic supports emerge to drive a learning design that meets 
the aim of student achievement above all. 


These strategic supports for the Design sub-dimension are: 


e Learner-Centered: Online offerings should adhere to best practices for user experience (UX) 
design, and should adhere to quality standards for how to best support learners as they navigate 
digital and online spaces. Content should be appropriately chunked to manage cognitive load, 
and should be engaging and include a variety of media types and modalities to fit learners’ needs, 
rather than simply to fit what educators have always done. 


e Research-Based: Educational research and inquiry continue to illuminate and inform best 
practices across contexts and sectors. Yet as Berliner (2002) has noted, educational research as a 
science deals with enormous complexities and challenges—all of which are centrally focused on 
human beings and how we teach them. In this way, the design of online K12 education moving 
forward should be informed and based on trusted and emergent educational research. 
Furthermore, a robust program of educational inquiry should accompany online and digital 
learning strategies, to contextualize and disseminate learnings within the Kingdom. 
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Project-Based: As Kereluik et al. (2013) and many others before have asked, what knowledge is of 
most worth?More and more, both educators and students are being pushed beyond the 
acquisition and demonstration of mere content knowledge in educational contexts, as the 
demand for transferability and applicability—or to put it flatly, relevance—in learning is at an all- 
time high. To help students develop the critical thinking, collaboration, and humanistic skills that 
are necessary to make their learning relevant beyond formal learning settings, project-based 
learning is essential. 


Engagement 

As educational opportunities expand through digital and online learning, emphasis should be placed on 
the ways in which educational experiences enliven students’ curiosities, interests, and commitments to 
their futures, as well as to the collective benefit of society. In this way, engagement is an integral 
component of the Teaching and Learning dimension. 


The strategic supports for the Engagement subdimension are: 


Personalization: Personalized learning opportunities create the space for learner autonomy and 
agency to drive their own learning. In personalizing pathways, learners can deepen their 
understanding of themselves as human beings, and experience learning that is relevant, 
challenging, contextualized, and designed for their needs (LEAP Innovations, n.d.). As learners 
gain confidence, they can also begin to challenge themselves and demonstrate their learning in 
multiple ways. 


Interaction: Learning is inherently a social enterprise, where communities of practice (Donner, 
2021; Wenger, 1998) can enrich and reinforce the power of learning experiences. This is 
particularly true in online environments, where the social presence (Richardson & Swan, 2003) of a 
course impacts students’ sense of perceived learning and satisfaction. With the expansion of 
social (and social media) integrations into learning applications and modalities, increased 
interaction supports learner engagement—and ultimately achievement. 


Support: From advance tutoring and feedback related to performance, to routine technical or 
navigation support for a course, the ways in which students are supported has tremendous impact 
on their engagement and achievement. Within this framework, centering the needs of students 
and how they can be supported is critical for overall success, which lies at this framework’s heart. 


Assessment 
Assessment of and for learning plays an important role within the practice of teaching and learning, and 
can be an overlooked area for innovation. 


The strategic supports for Assessment are: 
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e Authentic: The days of standardized testing and skill-and-drill style assessments must come to 
end, as we move into the second quarter of the 21* century. With advancements in technology 
and delivery capability, assessments must be tied to how students share and demonstrate their 
knowledge—hopefully for some social or broader impact than just to satisfy a teacher’s 
requirements. Grading systems like mastery grading or even ungrading can create powerful 
learning experiences and remove the fear of compliance for students who will be counted on to 
contribute toward cognitively and socially complex challenges as adults. 


e Universal Pathways: Universal pathways are an emerging idea (see: Casey & Patrick, 2020) that 
seek to “develop each individual’s agency and ability to navigate pathways that will cross multiple 
sectors and institutions over the course of their lives.” Universal pathways take the notion of a 
standard curriculum, and its assessments, and create unique flexibility for the Aowand the what 
of delivery. These pathways seek to support balanced systems of assessments to support learning 
and recognize competencies, rather than standardized tests. 


Continuous Improvement 


As online and digital learning expand across the nation, it will be critical for leaders, as well as 
stakeholders at every level, to have a sense of the success and need for improvement with the K-12 
education sectors. However, this dimension takes on the name Continuous Improvement as a key 
conceptual cornerstone for the notion that no measures or indicators of success will ever be final or 
complete. Just as economies and social institutions are a continual work in progress across a nation, so 
too should its system of K12 education be. And as digital and online learning continue to expand, this will 
become even more important as a driver of innovation and service delivery. 


Effectiveness 

The stakes for successful provision of educational services and student achievement have never been 
higher, particularly on a global scale. To this end, the effectiveness of educational service delivery must be 
a nationwide priority for K-12 education. Within this subdimension, and emphasis on networked learning 
and broad benchmarking for effective practices should become the new norm for online learning across 
the Kingdom of Saudi Arabia. 


The strategic supports for the Effectiveness subdimension are: 


e Improvement Science: The emerging field of improvement science (Bryk et al., 2015; Lewis, 2015) 
offers an enlightened approach to traditional measurement and evaluation that empowers 
practitioners who are subject to such measurement and evaluation to participate in the 
improvement process. As the recent investment by the Bill and Melinda Gates Foundation into 
networked improvement communities (NICs) suggests, a powerful method for supporting systems 
change and improvement comes from cyclical, action-oriented approaches to data collection and 
innovation efforts, led in part by participants themselves. A nationwide effort to create NICs at 
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scale would create dynamic learning opportunities and robust datasets for evaluation that 
perhaps no other nation has dared to undertake. 


e Benchmarking: While traditional rank-order data collection and evaluation has long, 
benchmarking practices and broad data sharing still has a valuable place. Benchmarking effective 
schools, teachers, leaders, and practices can inform the development of supports where needed, 
as well as propel successful innovations. As the Madrasati benchmarking component of this report 
has shown, having contextual knowledge of the landscape is a critical factor for success in large- 
scale systemic efforts. 


Learner Experience 

With respect to performance evaluation, schools and systems of education can often yield volumes of 
data to be analyzed. Yet, the experiences of the learners may be a space where ministries of education can 
do more to understand the impact and efficacy of their service delivery. To this end, the Learner 
Experience subdimension seeks to account for the ways in which broad systems of data collection, 
tracking, and analysis might more deeply target creating profiles and rich understandings’ of students 
experiences as a way to support student achievement. 


Strategic supports for the Learner Experience subdimension include: 


e Reimagined Evaluation: Schools can no longer afford to be designed around the notion of the 
average student and conducted as one-size-fits-all approaches to mass delivery (Rose, 2016). To 
pay credence to the notion that each student is unique and has unique potential to succeed, 
education systems must be responsive to their individual needs and provide opportunities for 
each student to develop and grow in their own way. To this end, efforts to personalize learning, 
both with respect to what and how they learn, will be essential. 


e Satisfaction: As this framework centers student achievement, it is perhaps fitting that a final 
dimension of this framework ought to relate to a commitment to provide those students with a 
satisfactory experience or set of experiences. Indeed, student feedback is often an overlooked 
component for improving the quality of educational experiences. In this way, emphasizing student 
satisfaction and seeking student input in systemic and structured ways will likely lead to more 
positive outcomes across the system as a whole, including increased student engagement, higher 
levels of student and instructor trust, and the ultimate goal of student achievement. 


Efficiency 

National efforts to promote innovation and realize the vision of student success for all will require broad 
commitment and effort across the other elements addressed through this framework. However, it will also 
be important to capture data and insights about the effectiveness of educational efforts and gauge the 
progress that students are making toward the ultimate goal of student achievement. To this end, the 
Efficiency subdimension supports the overall Continuous Improvement dimension. 
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Strategic support for the Efficiency subdimension includes: 


e Student Success & Growth Measures: While other elements of this framework promote 
assessment and teaching and learning practices that move beyond mere standardized 
assessments, there is still value and much to be learned from international large-scale 
assessments (ILSAs), including the PISA, and. As Singer et al. (2018) shared, such large-scale 
assessments, as well as other summative assessment measures, allow ministries to describe and 
compare student achievement and examine relevant contextual factors; track changes over time 
in student achievement; disturb complacency about a nation’s education system and to spur 
educational reforms; create de facto international benchmarking by identifying top performing 
nations and jurisdictions; eva/uate the contextual effectiveness of curricula, instructional 
strategies, and educational policies, and exp/ore causal relationships between contextual factors 
(p. 70-71). In this way, such assessments play a valuable, contextualized role for illuminating other 
innovation efforts, and provide insights for future strategic directions across all three dimensions 
of the framework. 
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Conclusion & 


Concluding Remarks on Online Learning 

Acommon challenge in online learning is the use and planning of teaching when time is flexible, in real- 
time and over time. Through an online readiness intervention to provide students and parents skills in 
managing their time, such as planning of time, scheduling of time for homework and live class meetings, 
time for connecting with teachers and classmates, and time management tools and techniques, students 
and parents will become more competent and satisfied with their experience. As teachers better 
understand how to use asynchronous and synchronous mediums of communication and interaction more 
effectively for their teaching goals through experience and professional development, satisfaction with 
time online and time management has the potential to improve. 
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Teachers, parents, and students can incorporate time management and technology skills building 
techniques to improve student outcomes and stakeholder satisfaction across the groups. 


With continued growth in the access of affordable laptops and Internet, student and parents can learn 
how to more successfully learn online and at a distance through an online learning readiness intervention, 
while teachers can be supported through new professionally development opportunities to help them 
harness the characteristics of a flexible learning model in better understanding temporal elements of 
online learning while creating engaging and fun experiences for students. 


Concluding Remarks on LMS Benchmarking 

Related to the Madrasati LMS, this report has sought to analyze and benchmark the considerations for and 
global adoption of learning management systems in K-12 school settings, relative to the KSA’s 
development and launch of its Madrasati e-learning platform. While the COVID-19 pandemic has 
undoubtedly disrupted traditional in-person learning and accelerated the need for high-quality online and 
blended learning options for billions of learners, there is evidence from KSA that Madrasati is a step in the 
right direction toward that aim. 


A review of the global landscape for LMS adoption shows that while more and more nations are offering 
cloud-based, widely available commercial solutions for schools and districts from their regional or 
national level authorities, there is still much discrepancy between what different nations are doing. 
Challenges related to technological infrastructure, resourcing, geographic and political challenges, and 
the still emerging development of e-learning capacity for various stakeholders present a variety of hurdles 
to those developing e-learning solutions - not to mention the disruption of the pandemic itself. 


Of the remedies that national governments and ministries of education have sought in effort to maintain 
continuity of educational and schooling services to learners across the regions, KSA’s Madrasati e-learning 
platform shows unique promise and a unique position to support learning for all of the country’s 
students. Indeed, at the time of this writing it is likely that the Madrasati LMS is the most widely deployed 
internally developed LMS in operation, in terms of number of users and available e-learning courses and 
content. In the first semester, the platform engaged more than 6 million users, including up to 99% of the 
Kingdom's learners and their parents. 


On the whole, the Madrasati platform compares favorably with contemporary commercial offerings 
distributed across the globe. Like the commercially available learning management systems, the 
Madrasati platform provides benefits to a variety of stakeholders, including learners, teachers, 
administrators, supervisors, and parents. Each has a unique user role and set of capabilities within the 
platform that contributes to the overall success of learners in KSA. The platform’s features and benefits 
also compare favorably with the overarching strategic considerations for LMS implementation, including 
alignment, cost, power and durability, support, reporting and analysis. 


What makes Madrasati unique in the contemporary global e-learning landscape in the K-12 education 
sector is that it is an internally developed LMS developed and launched to its national audience, 
supporting KSA’s standards-based curriculum. Moreover, the platform /aunched in the fall of 2020, at a 
time when the constraints and challenges of the pandemic ensured that its introduction would be subject 
to maximum use and strain. While we do not have a full picture of the system’s capability and efficacy, the 
context for the development and launch alone suggest that the Madrasati platform has already 
accomplished something special. 
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Future possibilities for the Madrasati e-learning include further support of learning technology integration 
(LTI) tools, similar to its interoperability with Microsoft Teams. These integrations could include 
partnerships with content providers and publishers, as well as with third party software providers of 
student engagement tools, content creation and curation, and plagiarism detection, for example. Across 
KSA, further development and distribution of access to high-speed internet and computing devices would 
also support both synchronous and asynchronous learning opportunities for learners across the nation. 


The future state of the pandemic, and for global life itself, is of course unpredictable. However, we can be 
sure that nations will continue to support and nurture their learners’ development and growth, 
particularly in the realm of e-learning. With the Madrasati platform, the Kingdom of Saudi Arabia is well- 
positioned to continue that effort. 
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Appendix A: 


K-12 Initiatives 
‘Dimension:Leadership 


Subdimensions 


Governance 


Strategies 


Policies 


Process 
Resource Allocation 


Staffing & Innovation 


Initiatives 


Provide specific and timely communication related to 
fluidity with COVID-19 and online learning 
Comprehensive strategic planning to account for any 
discrepancies or disparities between planning goals and 
organizational alignment 

Policies that meet the needs of quality and innovative 
online teaching and learning 

Processes that clarify teacher performance evaluation 
Explore opportunities for equitable resource allocation 
for schools 

Processes that incentivize teacher participation in 
training and professional learning for online teaching 
Create pathways for innovation in online teaching and 
learning 


Achievement 
Level 
Done 


Done 


In Progress 


Done 
In Progress 


Done 


In Progress 


Subdimensions 


Instructional Design 
Methods & Universal 
Design for Learning 

(UDL) 

Learning Objectives 


Alignment 


Course Materials & 
Content 


Innovation 


Initiatives 


Embrace instructional design practices like UDL explicitly 
in the design of curricula 


Create opportunities for student participation in 
meaning-making around course learning objectives, and 
to articulate what success means in a way that they can 
understand and begin to own 

Teachers clarify learning outcomes that align with 
guidelines regarding standards for instructional planning 
Regularly review course materials and content 
Regularly update course materials and content with 
current and media-rich sources 

Encourage teachers to be more pedagogically or 
technologically innovative 


Achievement 
Level 
Done 


Done 


In Progress 


In Progress 
In Progress 


Done 
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Achievement 


Subdimensions Initiatives 
Level 
Communication 14 Establish expectations for responsiveness Done 
Engagement 15 Implement both formal and informal opportunities for Done 
students to interact with each other and the teacher 
Expectation Setting 16 Develop, define, and communicate expectations for Done 
student activity 
Outcomes 17 Develop and implement student learning outcome In Progress 
metrics for online courses 
18 Communicate findings, including onsite comparisons as In Progress 
appropriate 
19 Clearly articulate criteria for assignments In Progress 
Course Interaction 20 Support teacher-to-student and student-to-student In Progress 
interactions 
21 Establish online presence of the educator Done 
Feedback 22 Develop and communicate methods for providing Done 
feedback, and set expectations for timing and 
explanations of how feedback is related to learning 
objectives 
Innovation 23 Explicitly encourage innovation through Done 
communications, activities, and professional 
development 
24 Develop and implement professional development Done 


opportunities on innovative 
teaching and student support 


Subdimensions 


Initiatives 


Achievement 
Level 


Assessment Strategies 


25 


Elaborate and implement subject-related 
communications to students regarding assessments, 
their objectives and overall learning outcomes 


Done 


Assessment Processes 


26 


Promote open dialogue with all members of the 
educational process 


Done 


27 


Ensure that assessments pursue generally accepted best 
practices and directly correlate to teacher expectations 


Done 


Assessment Methods 


28 


Use diverse assessment approaches and techniques to 
better meet various students’ needs 


Done 


Innovation 


29 


Encourage innovative assessment through 
communications, activities, and related professional 
development 


In Progress 


30 


Develop and implement professional development 
opportunities on innovative (ICT- based) assessment 


Done 
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Subdimensions Initiatives Achievement 
Level 
Centralized Online 31 Identify schools that have not yet implemented online In Progress 
Education Infrastructure learning and provide a centralized LMS with SSO 
Operability 32 Develop a feedback loop to ensure parent and student Done 
needs are met 
33 Create centralized resources that are comprehensible Done 
and policy accessible for parents and students 
Modality _34 Evaluate student experience over time Done 
35 Reflect on course design and areas for improvement Done 
based on needs 
Security 36 Develop a clear data security plan and share with Done 
stakeholders 
37 Develop policies around the use of third-party Done 
applications 
Information Technology 38 Determine IT pain points among stakeholders In Progress 
Service Management 39 Evaluate current processes, plans and resources Done 
(ITSM) Compliance 40 Develop new resources to address pain points Done 
Internet Access 41 Identify student internet needs Done 
42 Develop a distribution plan to share mobile hotspots Done 
43 Evaluate course content to ensure options for students to Done 
continue work with poor internet 
Reliability 44 Develop processes for monitoring and addressing the Done 
reliability of infrastructure and learning technology 
systems and courses 
Coverage 45 Stakeholders have multiple ways to report needs Done 
46 Periodic follow-ups and opportunities to acquire new Done 
technologies 
Innovation 47 Curriculum modification to include gaining experience Done 
working with student data at different levels of 
granularity 


Support 


Equity 


Accessibility 


Student Orientation and 


Compliance Standards 


Subdimensions 


48 


Initiatives 


Develop and implement comprehensive online course 
orientations 

Develop sufficient academic support resources 
Implement digital access for academic support resources 
Develop and implement appropriate evaluations to 
assess student support needs 

Develop and implement student support solutions that 
may be accessed at a distance 

Review the use of accessibility standards in online 
courses and ensure that gaps are addressed 

Review online courses for compliance with accessibility 
standards 


Achievement 
Level 

Done 

Done 

In Progress 

Done 

Done 


Done 


Done 
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Social Emotional 55 Develop and implement appropriate evaluations to Done 
assess social emotional student support needs 
56 Develop and implement social emotional student support Done 
solutions that may be accessed at a distance 


57 Develop and implement opportunities for students to Done 
build relationships and communities within their courses 
Innovation 58 Discover and implement innovative methods of Done 


communicating with students in online courses 


Achievement _ 


Subdimensions Initiatives 
eaniciedane, ___Level_ 
Technical Assistance 59 Create a centralized support space through a Done 
website/education portal 
60 Resources explicitly shared and stressed by teachers in Done 
syllabi and online course, and school 
Professional 61 Continuing professional subject- and technology-related Done 
Development training and support for teachers to develop the 


knowledge, skills, and abilities essential for designing and 
giving online classes 

62 Extend and reinforce teachers’ professional advancement Done 
and promote educators’ capacity building 


Orientation 63 Develop, test and share common orientation modules for Done 
schools 

Mentoring 64 Design a virtual peer support (peer observation/peer Done 
evaluation) and mentoring program for collegial learning 
groups 

Innovation 65 Develop a teacher learning community program In Progress 


Achievement 


Subdimensions Initiatives 
Level 
Evaluation of Course 66 Processes, procedures, and policies on course and Done 
Outcomes & Program program evaluation, including review and updating, are 
Quality developed and implemented 
Student Satisfaction 67 Create acommunication plan for students and parents Done 


regarding evaluation opportunities, purposes, and uses 
68 Show students and parents how their feedback has helped Done 
to improve the online learning courses or program 


Teacher Satisfaction 69 Create a communication plan for teachers regarding Done 
| evaluation opportunities, purposes, and uses 
Staff Satisfaction 70 Create acommunication plan for staff regarding Done 
evaluation opportunities, purposes, and uses 
Innovation 71 Implement best practices identified by online learning Done 
experts 
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Appendix B: 
List of National LMS Providers 


Internal vs. Fetinated 
Country LMS / platform commercial Other supports 


development PARE 


Argentina Educ.ar Internal Not available | Television broadcasts; 
radio broadcasts 


Austria Moodle; Eduthek Commercial; Not available | Digital content 
Internal, repositories 
developed with 
commercial 
vendors 


Bhutan Bhutan e-Learning | Internal Not available | Digital resources and 
platform textbooks 


China Empower Learning; | Internal, Not available | Television broadcasts; 
Educloud developed with online content 
commercial repositories 
vendors 


Colombia Aprender Digital Internal Not available | Television broadcasts; 
radio broadcasts 


Cote D’lvoire | Mon Ecole ala Internal Not available | Television broadcasts; 
Maison (“My School radio broadcasts 
at Home”) 

Egypt Edmodo Commercial 22 million SIM cards available free of 
charge for mobile users; 
television broadcasts; 
OER content repositories 


Greece Mathainoume- Internal 9.4 Million Television broadcasts 
stospiti 


MEHE LMS Not available | Television broadcasts 
In development Not available | Television broadcasts 


DIKSHA; Internal Not available | Television broadcasts; 
e-Pathshala; online content 
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South Korea 


Trinidad and 
Tobago 


United Arab 
Emirates 
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Libyan e-Learning Internal Not available 
Portal 

Aprende 2.0 Not available 
Myanmar Distance | Internal Not available 
Education Platform 

(MDEP) 

Epodreczniki.pl Not available 


REB e-learning Internal Not available 
platform 
Moja Skola (“My Internal Not available 
School”) 


EDUNET Internal 5.4 Million 
(e-Hakseupto & 

Waedorang as 

LMSs) 

Ministry of Internal, with Not available 
Education Online Microsoft 365 

Learning Platform login 

EBA (Egitim Bilisim | Internal 18 Million 
Agi) 

Learning Curve LMS Not available 
Tap Huan LMS Not available 


repositories 


Television broadcasts 


Television broadcasts; 
radio broadcasts; online 
content repositories 


Television broadcasts; 
YouTube channel 


Television broadcasts; 
digital resources and 
textbooks 


Radio broadcasts; digital 
resources and textbooks 


Television broadcasts; 
radio broadcasts 


Educational Broadcast 
System (TV); digital 
textbooks; 


Television broadcasts 


Television broadcasts; 
digital resources 


Access to Madrasa.org, an 
open educational 
platform 


Commercial provider: 
free integration w/Google 
Suite enterprise (S) 


CONSORTIUM. 
Appendix C: 
Analysis of National K-12 eLearning Solutions 


O O 
a @ = QO @ < 
e @ < @ O < 
. No | No | No | No | Yes : Yes | No | No | No | Yes 
Afghanistan Latvia 
; Yes | No | Yes | Yes | Yes No | No | No | No | No 
Albania Lebanon 
: Yes | No | Yes | Yes | Yes No | No | No {| No | No 
Algeria Lesotho 
Andorra No | No | No | No | No Liberia No No | No | No | Yes 
No | No | No | No | Yes : Yes | No | Yes | No | Yes 
Angola Libya 
Anguilla Yes | No | No | No | No Liechtenstein No No | No | No | No 
Antigua and Yes | No | No | No | No Yes | No | No | No | No 
Barbuda Lithuania 
; Yes | No | Yes | No | Yes Yes | No | No | No | No 
Argentina Luxembourg 
Armenia Yes | No | Yes | Yes | No Madagascar Yes | No | No | Yes | No 
Aruba No | No | No | No | No Malawi Yes | No | No | No | No 
Australia Yes | No | Yes | No | No Malaysia Yes | No | Yes | No | Yes 
Austria Yes | No | Yes | No | No Maldives Yes | No | No | Yes | Yes 
Azerbaijan No | No | Yes | No | No Mali Yes | No | No | No | Yes 
Bahamas Yes | No | Yes | No | No Malta Yes | No | Yes | No | Yes 
. Yes | No | Yes | No | No Malet No | No | No {| No | No 
Bahrain Islands 
Bangladesh No | No | Yes | No | No Mauritania Yes | No | No | No ]| Yes 
Barbados No | No | No | No | No Mauritius No | No | Yes | No | Yes 
Belarus No | No | No | No | No Mexico Yes | No | Yes | No | Yes 
Belgium No | No | Yes | No | No Micronesia No | No | Yes | Yes | No 
Belize No | No | Yes | No | No Moldova Yes | No | Yes | Yes | Yes 
Benin Yes | No | No | No | No Monaco Yes | No | No | No | No 
Bermuda No | No | No | No | No Mongolia No No | Yes | Yes | Yes 
No | No | No | Yes | Yes Yes | No | Yes | Yes | Yes 
Bhutan Montenegro 
_ No | No | No | No | Yes Montserrat No | No | No | No | No 
Bolivia 
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Bosnia and 
Herzegovina 


Botswana 


Brazil 


British Virgin 
Islands 


Brunei 
Darussalam 


Bulgaria 


Burkina Faso 


Burundi 


Cabo Verde 


Cambodia 


Cameroon 


Canada 


Cayman Islands 


Central African 
republic 


Chad 


Chile 


China 


Colombia 


Comoros 


Congo 


Congo DR 


Cook Islands 


Costa Rica 


Cote d'ivoire 


Croatia 


Cuba 


Curacao 


Cyprus 


No 


No 


No 


No 


No 


Yes 


No 


No 


No 


No 


No 


Yes 


Yes 


No 


Yes 
No 


Yes 


Yes 
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No 


No 


No 


No 


No 


No 


No 


No 


No 


No 


No 


Yes 
No 


No 


No 
No 


Yes 


Yes 


No 


No 


Yes 


No 


No 


Yes 


No 


No 


Yes 


No 


No 


Yes 


Yes 


No 


No 


Yes 


Yes 


Yes 


No 


No 


No 


No 


No 


No 


Yes 


Yes 


No 


No 


Yes 


No 


No 


No 


Yes 


No 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


No 


Yes 


No 


No 


No 


Yes 


Yes 


No 


Yes 


No 


Yes 


No 
No 


Yes 


No 
No 


Yes 


Yes 


Yes 


Yes 


Yes 


Morocco 


Mozambique 


Myanmar 


Namibia 


Nauru 


Nepal 


Netherlands 
New 
Caledonia 


New Zealand 


Nicaragua 


Niger 
Nigeria 
Niue 

North 
Macedonia 


North Korea 


Norway 


Oman 


Pakistan 


Palau 


Palestinian 
Territory 


Panama 
Papua New 
Guinea 


Paraguay 


Peru 


Philippines 


Poland 
Portugal 


Puerto Rico 


Yes 


Yes 


Yes 


Yes 


No 


Yes 


Yes 


No 


Yes 


No 


Yes 


Yes 
No 


No 


Yes 


Yes 


No 


No 


No 


No 


No 


No 


No 


Yes 


No 


Yes 


No 


No 


Yes 


No 


Yes 


Yes 


No 


No 


Yes 
No 


No 


No 
No 


Yes 


Yes 


No 


Yes 


Yes 


Yes 


Yes 


No 


No 


Yes 


No 


No 


Yes 


No 


Yes 


No 


No 


Yes 


Yes 
No 


No 


No 
No 


Yes 


No 


No 


Yes 


Yes 


No 


Yes 


Yes 


Yes 


Yes 


Yes 


No 


No 


Yes 


Yes 


No 


Yes 


No 


No 


Yes 
No 


No 


Yes 
No 


Yes 


Yes 


Yes 


Yes 
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Czechia 


Denmark 


Djibouti 


Dominica 


Dominican 
Republic 


Ecuador 


Egypt 


El Salvador 


Guinea 


Equatorial 


Eritrea 


Estonia 


Eswatini 


Ethiopia 


Fiji 


Faroe Islands 


Finland 


France 


French Guiana 


French 


Polynesia 


Gabon 


Gambia 


Georgia 


Germany 


Ghana 


Gibraltar 


Greece 


Greenland 


Grenada 


Guadeloupe 


Guam 


No 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


No 


No 
No 
No 
No 
No 


Yes 


Yes 


Yes 


No 


No 


Yes 


No 


Yes 


No 


No 


Yes 


No 


No 
Yes 
Yes 


Yes 


No 


No 


No 


No 


No 


No 


No 


No 


No 
No 
No 
No 
No 


No 


No 


No 


No 


No 


No 


No 


No 


No 


No 


No 


No 


No 
No 
No 
No 


oc 
cs 
fe 
fe 
rele 
Yes | No 
oo 


No 


No | No 


No 


No 


[ne 
Tes | Ne 
cc 


| No | 
Ic 
| No | 


Yes | No 


Yes | No 


es [no | 
oc 
re 


No 


Yes 


No 


Yes 


No 


No 


No 


Yes 


No 
No 
No 
No 
No 


Yes 


No 


Yes 


No 


No 


No 


Yes 


Yes 


No 


Yes 


No 


Yes 


No 
No 
No 
No 


Qatar 
Republic of 
Korea 


Romania 
Russian 
Federation 


Rwanda 


Saint Kitts and 
Nevis 


Saint Lucia 


Saint Vincent 
and the 
Grenadines 


Samoa 

San Marino 
Saudi Arabia 
Senegal 
Serbia 


Seychelles 


Sierra Leone 


Singapore 


Sint Marteen 


Slovakia 


Slovenia 
Solomon 
Islands 


Somalia 


South Africa 
South Sudan 


Spain 

Sri Lanka 

Sao Tomé and 
Principe 
Sudan 
Surinam 
Sweden 


Switzerland 


Yes 


Yes 


Yes 


Ye 


wn 


Ye 


wn 


Ye 


wn 


N 


°o 


N 
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Yes 


No 


No 


Yes 


Yes 


fmf 
Sno 
pepe l= 
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No 


No 


No 


No 


SOG 
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Sooo 
Sooo 


[Yes | 
[Yes | 


Yes 


Yes 


[ No | 
[No | 


No 
No 


[No | 


No 
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Guatemala 


Guinea 


Guinea-Bissau 


Guyana 


Haiti 


Honduras 
Hong Kong 
(SAR) 


Hungary 


Iceland 


India 


Indonesia 


Iran 


Iraq 


Ireland 


Israel 


Italy 


Jamaica 


Japan 


Jordan 


Kazakhstan 


Kenya 


Kiribati 


Kuwait 


Kyrgyzstan 


Lao PDR 
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Yes 


Yes 


No 


No 


No 


Yes 


No 


No 


No 


No 


Yes 


No 


No 
No 


Yes 


Yes 


No 


Yes 


Yes 


Yes 


Yes 


No 
Yes 
Yes 

No 


No 


Yes 


No 


No 


No 


No 


No 


No 


No 


No 


No 


No 


No 
No 


No 


No 
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About NELC About OLC 
The National eLearning Center was _ Established in 1999 by the Sloan 


established as an independent Foundation, the Online Learning 
entity by the Council of Ministers of | Consortium is a collaborative 
Saudi Arabia, aims to enhancing community of higher education 
trust in the eLearning programs, leaders and innovators, dedicated 
Leading innovation in Learning to advancing quality digital 


digital transformation and enabling teaching and learning experiences 
the integration among educational designed to reach and engage the 
institutions and labor market modern learner - anyone, anytime, 
needs. anyplace. 
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Grants, Online Learning Consortium 
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Learning Research and Development, and 
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for Distance Education and Technological 
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‘Clark Shah-Nelson, D.B.A. 
Assistant Dean, Instructional Design and 
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